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ABSTRACT 


Improved  Problem  Formulation 
In  Engineering  Systems  Design.  (May  1990) 
Charles  Walter  Ennis,  Jr., 

B.S.,  United  States  Military  Academy; 

M.S.,  University  of  Michigan 
Chair  of  Advisory  Committee:  Dr.  Steven  W.  Gyeszly 


The  objective  of  this  work  was  to  improve  problem  formula¬ 
tion  in  engineering  systems  design.  Exploratory  research 
investigated  when,  how  and  why  information  and  knowledge  are 
introduced  into  engineering  systems  design.  The  investiga¬ 
tion  employed  surveys  of  practicing  engineers  and  protocol 
analysis  of  expert  designers  in  controlled  task,  free 
response  experiments.  Based  on  exploratory  findings,  a 
method  to  improve  problem  formulation  was  developed.  The 
effectiveness  of  the  method  was  validated  with  a  controlled 
task,  guided  response  experiment  using  a  Multiple  Attribute 
Value  model  for  evaluation.  Performance  Identification  and 
Performance  Measurability  were  defined  as  two  essential 
attributes  for  assessing  improvements  in  problem  formulation. 
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1  INTRODUCTION 


Engineering  systems  design  is  a  decision-making  process 
through  which  an  engineer  creatively  applies  knowledge  to 
information  to  develop  a  product  or  process  that  will  satisfy 
performance  objectives  within  a  specified  environment.  The 
quality  and  efficiency  with  which  an  engineeering  problem  is 
solved  is  linked  in  part  to  its  formulation  [1,2].  Volkema 
[3]  clearly  describes  the  importance  of  problem  formulation 
to  planning  and  design: 

It  is  not  difficult  to  make  a  cogent  argument  for  more 
investigation  into  the  problem  formulation  process. 
Because  problem  formulation  often  occurs  in  the  early 
stages  of  planning  and  design,  it  has  the  potential  for 
affecting  the  direction  of  all  succeeding  stages.  This 
is  due,  at  least  in  part,  to  the  strong  relationship  that 
exists  between  the  representation  of  a  problem  and  the 
domain  of  solutions  and  ideas  that  the  representation 
can  produce. 

When  a  problem  is  first  discovered,  it  is  rich  in 
solution  possibilities.  As  the  process  continues, 
however,  assumptions  and  constraints  are  added  in  an 
effort  to  bring  manageability  and  closure  to  the  problem. 
These  limit  the  scope  of  the  problem  and  the  range  of  the 
possible  solutions. 

Because  the  amount  of  information  needed  to  change  a 
decision  is  much  greater  than  the  amount  needed  to  make 
it  initially,  reformulation  of  a  problem  becomes  less 
likely  once  a  particular  formulation  is  selected  and 
pursued.  This  places  added  pressure  on  decision  makers 
to  avoid  premature  closure  and  to  select  "optimal" 
problem  statements.  A  problem  that  is  defined  with 
incorrect  presumptions  concerning  needs  and  opportunities 
can  result  in  significant  monetary  losses  as  well  as 
problem  solving  ineffectiveness. 


Journal  model  is  ASME  Journal  of  Mechanical  Design. 
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1.1  Research  Objective  and  Methods 

The  objective  of  this  research  project  was  to  improve 
problem  formulation  in  engineering  systems  design.  Research 
was  conducted  in  two  phases. 

Phase  one  investigated  when,  how  and  why  information  and 
knowledge  are  introduced  into  the  design  process  by  analyzing 
current  practices  in  an  engineering  systems  design  environ¬ 
ment.  Data  was  collected  using  two  techniques:  surveys  of 
practicing  engineers  and  protocol  analysis  of  a  controlled 
task,  free  response  experiment.  Analysis  of  the  survey  data 
consisted  of  one-way  and  two-way  contingency  tables  and 
graphic  representations.  Analysis  of  the  protocols  involved 
comparison  of  encoded  events  related  to  information  acquisi¬ 
tion  for  six  experts  solving  three  design  problems.  Phase 
one  is  discussed  in  Chapter  3. 

Based  on  the  findings  from  phase  one,  the  objective  of 
phase  two  was  to  develop  and  validate  a  method  for  improving 
problem  formulation.  The  method,  described  in  4.1,  uses  five 
dimensions  to  assist  in  identifying  boundary  and  performance 
criteria,  and  technical  performance  requirements  as  a  means 
of  expressing  them.  The  method  was  tested  in  controlled 
task,  guided  response  experiments,  as  discussed  in  4.3.  The 
results  indicate  that  defining  required  performance  and 
boundaries  in  measurable  technical  terms  improved  the  problem 
statements  from  which  solutions  can  be  developed. 
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1.2  The  Design  Process  and  Systems  Design 

Design  is  "the  decision-making  process  which. .. engineers 
use  in  the  formulation  of  plans  for  the  physical  realization 
of  machines,  devices  and  systems."  [4]  The  Accreditation 
Board  of  Engineering  Technology  (ABET)  definition  provides 
more  focus  in  stating  that: 

Engineering  design  is  the  process  of  devising 
system,  component,  or  process  to  meet  desired  needs.  It 
is  a  decision-making  process  (often  iterative) ,  in  which 
the  basic  sciences,  mathematics  and  engineering  sciences 
are  applied  to  convert  resources  optimally  to  meet  a 
stated  objective.  Among  the  fundamental  elements  of  the 
design  process  are  the  establishment  of  objectives  and 
criteria,  synthesis,  analysis,  construction,  testing  and 
evaluation.  [5] 

The  design  process  has  been  described  in  a  variety  of 
manners  throughout  its  evolutionary  history.  The  process  has 
been  divided  into  as  few  as  three  and  as  many  as  nine  or  more 
steps  or  phases  by  various  authors  [4,6,7,8,9,10,11].  Early 
approaches  to  design  methodology  attempted  to  force  designers 
to  follow  rigid,  lockstep  procedures  [6,7].  These  models 
have  given  way  to  approaches  which  recognize  a  need  for  the 
method  to  adapt  to  the  designer,  not  vice  versa  [12,13,14]. 
Second  generation  approaches  emphasized  design  as  a  par¬ 
ticipatory  or  argumentative  process  without  the  need  for 
rigid  procedures  [13,14].  A  third  generation  conjectures  a 
designerly  way  of  thinking,  recognizing  the  ill-defined 
nature  of  design  problems  and  that  the  designer's  activity 
oscillates  between  emerging  requirement  ideas  and  developing 
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solution  ideas  [11,13,15].  Regardless  of  the  number  of  steps 
or  the  degree  of  orderliness  involved,  design  is  a  decision¬ 
making  process  which  begins  with  some  form  of  need  iden¬ 
tification  which  is  translated  into  a  problem  statement  and 
then  progresses  in  a  generally  iterative  or  cyclic  manner  to 
generate  alternatives  and  select  a  solution  to  the  stated 
problem. 

A  system  may  be  defined  as  "a  set  of  resources — people, 
facilities,  equipment,  data  and  so  on — organized  to  perform 
designated  functions  to  achieve  desired  results."  [8]  A 
system  is  "assumed  to  be  an  entity  separable  from  the  rest 
of  the  universe  (the  environment  of  the  system)  by  means  of 
a  physical  or  conceptual  boundary."  [16] 

A  systems  approach  to  design  recognizes  that  a  product  or 
process  must  be  designed  to  operate  within  its  intended 
environment.  "The  essence  of  what  is  considered  to  be  the 
'systems  viewpoint'  is  to  concern  oneself  with  the  operation 
of  a  complete  system  rather  than  the  operation  of  the 
component  parts."  [16] 

Systems  approaches  emphasize  looking  at  the  problem 
beyond  its  existing  framework;  they  search  for  what  the 
purposes  or  problem  ought  to  be.  They  challenge  hidden 
assumptions  and  look  for  contextual  purposes.  The 
conventional  approach's  emphasis  is  on  analysis,  which 
is  onlv-  a  limited  part  of  design....  [Systems  approaches] 
increase  the  probability  of  working  on  the  right  problem. 
[17] 
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2  LITERATURE  REVIEW 

An  investigation  of  information  requirements  and  problem 
formulation  in  engineering  systems  design  requires  contribu¬ 
tions  from  several  disciplines,  to  include  design  methodol¬ 
ogy,  systems  methodology,  and  quality  engineering.  Each  area 
will  be  addressed  separately. 

2 . 1  Design  Methodology 

Design  methodology  is  the  broad  and  general  study  of  design 
principles,  practices  and  procedures,  with  a  focus  on  how 
design  both  "is"  and  "might  be"  conducted  [18].  The  majority 
of  research  on  the  design  process  "emphasizes  data  analysis, 
model  building,  selection,  detailing  and  post-implementation 
analysis.  Very  little  is  stated  about  methods  and  techniques 
for  the  important  steps  of  problem  finding,  formulation  and 
redefinition."  [19]  Traditional  texts  on  the  engineering 
design  process  [4,6,20,21]  typically  devote  between  one  para¬ 
graph  and  six  pages  to  the  transition  from  needs  identifica¬ 
tion  to  problem  statement. 

As  exceptions  to  the  general  emphasis  on  the  later  phases 
of  the  design  process,  Sakman  [22]  evaluated  the  effect  of 
three  explicit  methods  of  problem  formulation  on  the  quality 
of  general  (nonengineering)  problem  statements  developed  by 
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small  groups.  Nadler  [19]  evaluated  the  effect  of  three 
general  methods  of  problem  formulation  on  the  quality  of 
problem  statements  and  on  design  performance  in  mechanical 
engineering  design.  Esterline,  et.  al.  [23]  applied  protocol 
analysis  to  capture  expertise  in  kinematic  design.  Their 
goal  is  a  problem-formulation  module  for  a  knowledge-based 
expert  system  for  mechanism  design.  Cross  [24]  offers  the 
Objectives  Tree  Method  for  clarifying  customer  needs  and  the 
Function  Analysis  Method  for  describ:  what  must  be  de¬ 
signed.  McMahon  [25]  describes  Structured  Boundary  Examina¬ 
tion,  a  four-step  process  to  redefine  the  problem,  or  needs 
statement,  by  equating  parts  of  speech  (verbs,  nouns  and 
modifiers)  to  aspects  of  design  (process,  results  and 
specifications) .  Smith  [26]  evaluates  case  studies  to  show 
that  understanding  the  real  customer  needs  and  associated 
performance  requirements  increases  efficiency  and  effective¬ 
ness  in  creative  product  design. 

2.2  systems  Methodology 

AT&T  Bell  Laboratories  developed  a  methodology  for  trans¬ 
lating  customer  requirements  into  technical  terms  and  then 
tracking  technical  requirements  throughout  the  systems  design 
process  [27].  In  computer  systems  development,  Smith  [28] 
demonstrates  that  early  prototyping  is  being  used  as  an 
effective  tool  in  defining  what  the  customer  needs  or  wants. 
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2.3  Quality  Sciences 

Traditional  approaches  to  quality  emphasized  product 
inspection  or  process  control  through  statistical  analysis 
[29].  Recently,  quality  has  taken  on  new  definitions  and 
roles,  with  increasing  emphasis  on  the  planning  and  design 
stages  of  the  product  life  cycle.  Crosby  defines  quality  as 
"conformance  to  requirements,  not  goodness."  [30,  p.  64] 
Juran  [31,  pp.  20-27]  defines  the  Juran  Trilogy,  consisting 
of  quality  planning,  quality  control  and  quality  improvement. 
Imbedded  in  quality  planning  are  techniques  for  determining 
customer  needs  and  translating  customer  needs  into  the 
producer's  or  supplier's  language  [31,  pp.  94-108],  Quality 
Function  Deployment  (QFD)  [32,33]  offers  a  set  of  interlock¬ 
ing  matrices  that  links  customer  needs  or  "Voice  of  the 
Customer"  (VOC)  to  design  and  manufacturing  requirements. 
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3  PRELIMINARY  INVESTIGATIONS 


The  objective  of  the  first  phase  of  the  research  was  to 
investigate  when,  how  and  why  information  and  knowledge  are 
introduced  into  the  design  process  by  analyzing  current 
practices  in  engineering  systems  design.  In  order  to  use 
surveys  and  protocol  analysis  for  data  collection,  an  example 
of  engineering  systems  design  was  selected  to  provide  a 
sample  space.  Packaging  systems  design  was  selected  for  two 
primary  reasons. 

First,  it  is  representative  of  systems  design  in  that  it 
involves  the  interaction  of  materials,  machines,  and  people, 
both  within  the  system  and  between  the  system  and  its 
environment.  Packaging  systems  are  the  combination  of  the 
package,  the  equipment,  and  the  operations  required  to 
facilitate  distribution  of  a  product  from  its  point  of  pro¬ 
duction  to  its  point  of  final  use.  Distribution  may  include 
packing,  handling,  loading,  transporting,  storing,  display¬ 
ing,  and  retailing  the  item.  The  primary  purpose  of  packag¬ 
ing  is  to  minimize  the  interaction  between  the  product  and 
its  environment  during  distribution. 

Second,  packaging  systems  comprise  a  broad  range  of 
commonly  understood  concepts,  so  the  results  of  the  research 
can  be  easily  understood  and  translated  to  other  design 
domains.  For  example,  the  materials  and  processes  used  in 
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packaging  are  similar  to  other  engineering  applications.  The 
environmental  conditions  that  must  be  examined  (e.g.,  shock, 
vibration,  electrostatic  discharge)  are  familiar  to  engineers 
in  several  disciplines. 

An  ancillary  reason  for  selecting  packaging  systems  is  the 
potential  benefit  that  could  result  from  any  improvement  in 
the  ability  of  designers  to  efficiently  use  the  resources 
involved,  including  both  materials  and  manpower.  Packaging 
is  a  major  consumer  of  materials,  especially  glass,  plastics, 
and  paper  products  [34].  In  addition,  over  one  million 
people  are  involved  in  packaging  operations,  making  it  one 
of  the  largest  industries  in  the  United  States  [35].  Thus, 
any  improvement  in  packaging  system  design  could  have  broad 
impact  on  industry  and  consumers. 

3.1  Survey  of  Practicing  Engineers 

Two  sequential  surveys  were  used  to  determine  how  practic¬ 
ing  engineers  identify  and  obtain  the  information  and 
knowledge  required  in  engineering  systems  design.  The 
decision  to  use  surveys  in  lieu  of  other  information  gather¬ 
ing  techniques  (e.g.,  observation  or  eyewitness  accounts, 
performance  tests,  written  tests,  or  review  of  existing 
documentation  [36])  was  based  on  efficiency  and  the  type  of 
information  being  sought.  Surveys  are  effective  in  obtaining 
information  that  can  be  categorized  as  attitudes,  beliefs, 
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behaviors  or  attributes  [37].  Information  in  each  of  the 
four  categories,  but  primarily  behaviors  and  attributes,  was 
required  for  this  research.  Surveys  are  also  efficient  in 
obtaining  information  from  a  large  population  in  a  short  time 
frame  and  at  a  reasonable  expense. 

Surveys  may  consist  of  questionnaires,  telephone  inter¬ 
views,  face-to-face  interviews,  or  combinations  thereof.  The 
mailed,  self-administered  questionnaire  was  selected  because 
of  the  large  potential  sample  size  and  because  it  permits  the 
participant  to  work  at  his  or  her  desired  pace.  Question¬ 
naires  do  not  easily  permit  in-depth  probes  or  follow-up 
questioning  of  individual  participants.  This  was  not 
considered  to  be  a  detriment  to  this  research  program  since 
primarily  behavioral  and  attribute  data  were  sought. 

3.1.1  Survey  Development  and  Distribution 

Numerous  references  offer  guidelines  for  writing  survey 
questions  [38,39,40],  administering  surveys  [36,38],  and 
survey  analysis  [41,42,43,44].  For  an  overall  approach  to 
surveying,  the  Total  Design  Method  (TDM)  [37]  was  selected 
because  it  follows  a  philosophy  similar  to  the  systems 
approach  to  engineering  design.  TDM  strives  to  maximize 
survey  effectiveness  by  first  identifying  each  aspect  of  the 
survey  process  that  may  affect  either  the  quality  or  quantity 
of  response,  then  developing  a  plan  to  minimize  any  negative 
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effects.  Instead  of  focusing  inward  on  the  details  of  survey 
preparation,  TDM  initially  focuses  outward  on  the  environment 
in  which  the  survey  must  be  successful.  Combining  TDM 
philosophy  with  techniques  of  [36,38,40,43]  resulted  in  a 
satisfactory  45%  response  rate  to  the  initial  survey  and  a 
94%  response  rate  to  the  follow-up  survey. 

Each  survey  was  developed  in  a  three-phase  process.  The 
first  phase  determined  the  survey  objectives  and  the  informa¬ 
tion  required  to  support  those  objectives.  The  second  phase 
was  to  determine  the  type  of  information  being  sought  and 
write  the  individual  questions  in  support  of  the  survey 
objectives.  The  final  phase  was  to  arrange  the  questions 
into  a  logical  format  which  encourages  response  and  minimizes 
participant  error. 

The  initial  survey  was  a  cross-sectional  (i.e.,  adminis¬ 
tered  only  one  time)  design  which  sampled  engineers  selected 
from  the  1989  membership  roles  of  the  Society  of  Packaging 
and  Handling  Engineers  (SPHE) .  Forty-five  replies  were 
received  from  the  mailing  of  one  hundred  initial  surveys. 
From  among  those  respondents  who  agreed  to  participate  in  the 
follow-up  survey,  33  were  selected  as  representative  of 
engineers  in  corporations  that  develop  packaging  systems  in 
support  of  their  product  lines.  There  are  no  uniform 
standards  for  determining  sample  size  for  surveys,  but  the 
general  rule  is  that  larger  samples  will  provide  higher 
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levels  of  accuracy.  When  used  in  conjunction  with  literature 
reviews  and  other  information  sources,  a  sample  of  20  to  50 
respondents  is  considered  appropriate  during  the  early  stages 
of  research.  The  small  sample  size  is  adequate  for  develop¬ 
ing  hypotheses  and  procedures  for  testing  them,  since  the 
primary  purpose  is  to  identify  major  problem  areas  and 
dimensions  of  the  problem  that  the  researcher  may  have 
ignored.  In  contrast,  if  the  purpose  of  the  survey  was  to 
draw  conclusions  that  would  have  significant  economic  or 
social  impact,  an  appropriate  sample  size  could  be  two  or 
three  orders  of  magnitude  larger  [43,  pp.  146-148]. 

3.1.2  Survey  Objectives 

The  objectives  of  the  initial  survey  were  (1)  to  identify 
and  collect  background  data  on  potential  participants  for 
follow-up  research,  and  (2)  to  identify  the  range  and  modes 
of  information  and  knowledge  used.  The  information  sought 
to  support  the  objectives  is  listed  in  Table  1  and  further 
detailed  in  Appendix  A.l. 

Initial  survey  responses  helped  formulate  topics  for  the 
follow-up  survey.  The  objectives  of  the  follow-up  survey 
were  to  (1)  determine  if  engineers  currently  apply  a  systems 
approach  to  engineering  design,  and  (2)  determine  what  effect 
a  systems  approach  has  on  the  identification  and  acquisition 
of  information  and  knowledge  during  the  design  process. 
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The  information  sought  to  support  the  objectives  is  listed 
in  Table  2  and  detailed  in  Appendix  B.l. 


Table  1.  Information  Sought  During  Initial  Survey 
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3.1.3  Survey  Analysis 

Hypothesis  testing  is  a  fundamental  tool  in  the  analysis 
of  statistically  designed  experiments  [45,46].  However,  in 
the  analysis  of  surveys,  hypothesis  testing  is  rarely  the 
sole  or  even  dominant  analysis  technique.  This  is  due  in 
part  to  the  uncontrolled  nature  of  surveys,  as  contrasted 
with  a  highly  controlled  laboratory  experiment.  Rossi  [43] 
offers  three  primary  reasons  for  this  argument.  First,  while 
survey  data  may  support  a  hypothesis,  it  cannot  prove  the 
theory  from  which  the  hypothesis  was  postulated.  The 
hypothesis  may  be  consistent  with  theories  other  than  the  one 
under  investigation.  Second,  strict  hypothesis  testing  does 
not  look  for  the  existence  of  extraneous  variables  which  may 
have  influenced  the  data.  Third,  the  survey  may  contain 
information  supporting  research  ideas  which  could  not  be 
stated  in  clearly  stated  a  priori  hypothesis.  This  informa¬ 
tion  can  be  extracted  using  techniques  described  as  elabora¬ 
tion,  the  flow  of  analysis,  the  "pursuit  of  an  idea",  and 
other  post  factum  interpretations  of  the  data  [43,  pp.  201- 
239]  . 

Survey  analysis  was  conducted  in  three  steps.  First,  the 
data  was  edited,  encoded  and  recorded,  [44,  pp.  100-113] 
using  a  personal  computer  spread  sheet,  PlanPerfect.  Second, 
preliminary  analysis  of  the  data  included  interpretation  of 
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one-way  and  two-way  contingency  tables,  or  graphs  of  frequen¬ 
cy  distributions.  Finally,  post  factum  analysis  searched  for 
information  that  could  lead  to  further  research  topics. 

The  type  of  data  solicited  and  the  structure  of  the  survey 
questions  limited  analysis  in  most  cases  to  subjective 
evaluation  of  contingency  tables  or  plots  of  frequency 
distributions  for  two  reasons.  First,  in  most  questions  the 
random  variables  belong  to  a  nominal  scale,  where  the  only 
characteristic  is  that  each  variable  is  distinguishable  from 
the  others.  The  mode  is  the  basic  statistical  measure  for 
evaluating  variables  on  a  nominal  scale  [8,  pp.  45-48],  and 
no  other  statistic  measures  are  applicable. 

Second,  in  those  cases  where  an  ordinal  or  interval  scale 
was  used  to  measure  a  variable  such  as  relative  importance 
or  relative  difficulty,  the  majority  of  the  survey  questions 
required  the  respondent  to  rate  all  applicable  topics.  Not 
rating  a  topic  indicated  a  response  of  "not  applicable", 
which  does  not  belong  on  a  scale  ranging  from  "not  difficult" 
to  "very  difficult".  Consequently,  the  sum  of  the  probabili¬ 
ties  for  all  possible  values  of  the  random  variable  along  the 
scale  does  not  equal  one,  a  necessary  condition  in  the 
definition  of  a  discrete  probability  distribution. 
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3. 1.3.1  Initial  Survey  Analysis 

The  initial  survey  with  response  data  is  Appendix  A. 2.  The 
initial  survey  respondents  represent  a  relatively  narrow 
range  of  personal  and  professional  attributes.  The  respon¬ 
dents  can  be  categorized  as  predominantly  engineers  with 
between  2  and  10  years  experience  in  packaging  or  packaging 
related  positions.  The  modal  respondent  is  between  26  and 
35  years  old  with  a  bachelor  degree  or  some  graduate  study. 
A  1988  salary  survey  of  packaging  professionals  [47]  sampled 
from  a  similar  population  describes  the  average  packaging 
professional  as  42  years  old  with  a  bachelor  degree  and  14.4 
years  of  packaging  experience.  Over  20%  of  the  respondents 
to  the  salary  survey  were  managers  of  packaging,  while  24% 
percent  were  packaging  or  project  engineers.  The  larger 
proportion  of  managers,  in  contrast  with  the  initial  survey 
for  this  research,  would  tend  to  increase  the  average  age  and 
experience  levels.  Based  on  this  comparison,  the  set  of 
respondents  to  the  initial  survey  is  assumed  to  be  represent¬ 
ative  of  an  average  engineer  in  the  packaging  profession. 

Almost  exclusively,  the  respondents  worked  at  division  or 
corporate  levels  in  a  company  employing  more  than  500  people. 
The  companies  manufactured  a  wide  variety  of  products  for 
consumer  or  industrial  application  and  used  packaging  systems 
as  part  of  the  production  and  distribution  process. 

The  areas  in  which  information  is  required  to  develop  new 
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packaging  systems  were  uniformly  distributed  among  the  eight 
topics  offered  in  the  questionnaire  with  three  additional 
topics  identified  by  respondents  (Fig.  1) .  When  arranged  in 
order  of  decreasing  difficulty  in  obtaining  the  required 
information,  the  top  five  were  topics  other  than  the  packag¬ 
ing  itself  (Fig.  2)  .  This  indicates  that  information  outside 
the  engineer's  immediate  area  of  expertise  is  more  difficult 
to  identify  and  obtain.  This  topic  was  selected  for  follow¬ 
up  investigation.  Sources  of  information  and  their  relative 
frequency  of  use  were  uniformly  distributed  among  five  of  the 
topics  listed  (Fig.  3) . 

Categories  of  knowledge  were  distributed  among  nine  topics, 
but  only  five  of  those  ranked  high  in  relative  frequency  of 
use  (Fig.  4)  .  Experience  on  the  job  and  formal  education 
were  bimodal  sources  for  acquiring  knowledge.  Analysis  of 
this  data  offered  no  insights  or  patterns  for  follow-up 
questions . 

Responses  to  questions  about  important  aspects  of  an  en¬ 
gineer's  job  indicated  that  interpersonal  skills,  especially 
the  ability  to  communicate,  were  as  important  as  technical 
skills.  This  topic  was  refined  to  focus  on  the  need  for 
effective  communication  in  determining  system  requirements 
and  selected  for  follow-up  investigation. 
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Figure  2.  Information  That  is  Most  Difficult  to  Obtain 
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Figure  3.  Sources  of  Information  Most  Often  Used 


Figure  4 


Forms  of  Knowledge  Most  Often  Used 
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3. 1.3. 2  Follow-up  Survey  Analysis 

The  follow-up  survey  with  response  data  is  Appendix  B.2. 
The  detailed  analysis  of  individual  questions  is  Appendix 
B.3.  One-way  and  two-way  contingency  tables  and  frequency 
distribution  plots  were  the  primary  analysis  tools.  Cor¬ 
relating  data  between  individual  questions,  three  general 
conclusions  were  made: 

1.  The  respondents  do  use  a  systems  approach  to  solving 
packaging  problems,  although  they  may  not  formally  recognize 
it  as  such. 

2.  Clearly  stated  technical  performance  requirements  are  not 
used  as  a  deliberate  part  of  the  design  process.  Translation 
of  requirements  into  technical  terms  appears  to  be  a  by-prod¬ 
uct  of  the  design  process. 

3 .  Information  and  knowledge  outside  the  immediate  area  of 
expertise  are  more  difficult  to  obtain.  Information  about 
marketing  is  the  one  area  which  combines  a  high  level  of 
importance  and  high  relative  difficulty  to  obtain  informa¬ 
tion. 


3.1.4  Summary  of  Survey  Results 

The  initial  survey  established  the  audience  and  provided 
two  indicators  of  topics  of  importance  which  were  pursued  in 
the  follow-up  survey  and  subsequent  experimentation.  The 
first  topic  was  the  premise  that  information  outside  the 
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engineer's  immediate  area  of  expertise  is  more  difficult  to 
identify  and  obtain.  This  was  included  in  the  follow-up 
survey.  The  second  topic  was  the  need  for  effective  com¬ 
munication  in  determining  system  requirements.  This  was 
pursued  in  a  design  exercise  focused  on  translating  marketing 
needs  into  design  requirements  [48]. 

The  follow-up  survey  provided  indications  that  the  respon¬ 
dents  did  apply  a  systems  approach  to  developing  solutions 
to  packaging  problems,  confirming  that  the  respondents  were 
an  appropriate  sample  for  this  research.  It  also  supported 
the  hypothesis  that  information  outside  the  immediate  area 
of  expertise  is  more  difficult  to  obtain.  It  failed  to 
support  the  hypothesis  that  engineers  deliberately  translate 
performance  requirements  into  technical  terms  during  the 
design  process. 

3.2  Protocol  Analysis  of  the  Design  Process 

The  design  processes  of  several  highly  experienced  and 
successful  packaging  systems  engineers  were  documented  and 
analyzed  to  search  for  commonalities  which  may  be  in  part 
responsible  for  their  success.  Protocol  analysis  was  used 
in  controlled  task,  free  response  experiments  to  capture 
verbal  reports  as  data.  The  experiments  were  focused  on  the 
phases  of  design,  beginning  with  the  identification  of  a  need 
and  concluding  with  the  generation  of  design  concepts  for 
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evaluation.  Emphasis  was  placed  on  what  information  the 
engineer  sought,  and  what  influence  that  information  had  on 
the  concepts  generated. 

3.2.1  Protocol  Analysis 

Protocol  analysis  is  the  best  technique  used  in  cognitive 
sciences  to  study  complex  behavioral  processes,  such  as 
problem  solving  [49].  Subjects  are  required  to  "think  aloud" 
as  they  solve  problems.  The  recorded  proceeding,  or  proto¬ 
col,  provides  a  detailed  log  of  the  subject's  activities. 
The  analysis  of  the  protocol  identifies  the  steps  taken  by 
the  subject,  and  can  provide  insight  for  in-depth  study  of 
segments  of  the  process  [50] .  Protocol  analysis  and  other 
methods  of  acquiring  verbal  data  about  thought  processes  are 
generally  accepted  techniques  for  generating  hypotheses  and 
ideas  for  further  exploration  [51,  pp.  2-4]. 

Protocol  analysis  offers  advantages  over  two  other  ap¬ 
proaches  to  examining  the  design  process.  Retrospective 
analysis  requires  the  participant  to  reflect  from  memory  what 
was  done  during  an  earlier  design  process.  The  disadvantages 
of  retrospective  analysis  are  (1)  loss  of  some  detail  due  to 
filtering  by  the  participant,  and  (2)  potential  for  biasing 
as  the  participant  reports  what  he  or  she  thinks  was  done  or 
modifies  actual  behavior  to  reflect  what  he  or  she  thinks 
should  have  been  done.  Informal  reporting  involves  the 
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observer  taking  notes  and  asking  questions  during  the  design 
process.  It  is  the  least  disruptive  but  captures  only  data 
requested  by  the  observer.  Protocol  analysis  provides  the 
most  detailed  and  unbiased  verbal  data  of  the  three  common 
techniques  One  drawback  of  protocol  analysis  is  that  it 
tends  to  slow  down  the  participant,  but  it  is  reported  to  not 
significantly  affect  the  order  or  content  of  the  design 
process  [51, pp.  104-107,52]. 

Three  major  studies  have  used  protocol  analysis  to  study 
the  design  process.  Adelson  and  Solway  [53]  studied  software 
engineers  in  the  design  of  an  electronic  mail  system.  Kant, 
Newell  and  Steier  [54,55,56]  studied  graduate  students 
developing  algorithms  for  computational  geometry.  Ullman, 
Stauffer,  Dietterich  and  McGinnis  [57,58,59,50,60,61]  studied 
six  mechanical  engineers  involved  in  conceptual  and  detailed 
design  of  mechanical  devices. 


3.2.2  The  Experiment 

In  controlled  task,  free  response  experiments,  the  par¬ 
ticipants  are  provided  a  fixed  task  but  no  guidance  or  rules 
on  how  to  complete  the  task  [62].  Appendix  C  describes  the 
instructions  provided  and  the  experimental  tasks.  Three  ex¬ 
perimental  tasks  were  developed  to  correspond  with  the  areas 
of  product  expertise  of  the  participants .  The  tasks  were 
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chosen  to  offer  realistic  technical  and  nontechnical  challen¬ 
ges  without  being  excessively  complex  in  scope. 

Six  industry  experts  were  selected,  two  each  from  three 
product  areas.  The  participants  ranged  in  experience  from 
15  to  43  years  as  packaging  system  developers.  All  had 
technical  educational  backgrounds  ranging  from  two  and  a  half 
years  of  college  to  master  degrees.  All  were  directors  or 
managers  of  packaging  functions,  currently  responsible  for 
development  of  packaging  systems  within  a  corporate  environ¬ 
ment.  Each  participant  was  selected  based  on  his  or  her 
professional  reputation  among  colleagues,  as  well  as  level 
and  breadth  of  experience. 

The  product  areas  were  chosen  as  representative  of  dif¬ 
ferent  types  of  challenges  facing  a  packaging  system  develop¬ 
er.  The  electronics  area  represents  a  high  cost,  low 
production  rate  item  that  is  sensitive  primarily  to  shock  and 
vibration,  but  experiences  little  interaction  with  the  sur¬ 
rounding  environment.  The  food  product  is  a  moderate  cost, 
moderate  production  rate  item  that  is  subject  to  interaction 
with  the  environment  that  must  be  controlled  primarily  to 
protect  the  product.  The  medical  product  represents  a  low 
cost,  high  production  rate  item  where  interaction  must  be 
controlled  to  protect  both  the  product  and  the  environment. 

Each  experiment  was  conducted  at  the  subject's  work 
location  to  allow  access  to  normal  references  and  design 
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tools.  The  participant  was  provided  general  instructions 
describing  how  the  exercise  would  proceed  (Appendix  C.l). 
After  recording  biographical  information,  the  participant  was 
provided  the  brief  description  of  the  product  to  be  packaged 
(Appendix  C.2).  Verbal  data  was  recorded  on  audio  tape.  All 
design  notes  and  sketches  were  collected  on  sequential  pages. 

3.2.3  Analysis  Procedure 

The  evaluation  of  the  protocols  consisted  of  three  general 
phases:  data  transcription  and  editing,  encoding,  and 
analysis.  Verbal  data  was  transcribed  verbatim,  then  edited 
to  remove  irrelevant  comments  and  information  identifying  the 
participant  or  company  involved.  Irrelevant  comments  (e.g. 
"I  need  to  take  a  break.")  were  removed  to  streamline  the 
analysis.  Identifying  data  was  deleted  to  maintain  confiden¬ 
tiality. 

The  protocols  were  reviewed  to  identify  the  introduction 
of  information  or  knowledge  into  the  design  process.  A 
coding  scheme,  shown  in  Table  3,  was  used  to  identify  five 
characteristics  about  each  occurrence.  To  reduce  the  pos¬ 
sibility  of  biasing  the  data,  each  protocol  was  independently 
encoded  by  two  mechanical  engineering  graduate  students,  each 
of  whom  was  involved  in  research  associated  with  the  en¬ 
gineering  design  process.  The  two  researchers  agreed  upon 
a  composite  set  of  coded  occurrences  to  be  used  for  analysis. 
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Appendix  C.3  is  a  sample  coded  protocol.  As  shown  in  Table 
4,  the  correlation  of  identifying  the  introduction  of 
information  and  knowledge  ranged  from  45%  to  65%,  calculated 
by  comparing  the  number  of  common  occurrences  before  com¬ 
parison  to  the  number  of  occurrences  by  consensus.  A 
majority  of  the  discrepancies  were  in  determining  whether  an 
occurrence  was  new  information  or  the  confirmation  or 
restatement  of  previous  information.  For  those  occurrences 
that  were  commonly  identified,  the  coding  correlation  ranged 
from  54%  to  96%,  generally  improving  with  repetition  of  the 
process.  Again,  a  consensus  for  each  code  was  entered  in  the 
composite  set. 


Table  3 .  Codes  for  Information  Occurrences 


Cl ass if i cat  ion 

Focus 

Level 

-- 

Terminology 

Concept 

R,H.U 

P,C,K,S,0,l,X 

G,S 

1 

Reauested 

Product 

General 

Mon-technical 

1  Concept 

data 

Component 

Specific 

Technical 

Introduced 

Historical  or 

assumed  data 

package 

knowledge  to 

process 

manipulate  data 

Distribution 

Environment 

Internal 

external 


J 
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Table  4 .  Correlation  of  Coded  Data 


NUMBER  OF 

HDTV 

SWAB 

PIZZA 

PIZZA 

HDTV 

SWAB 

OCCURRENCES 

A 

B 

C 

D 

E 

F 

RESEARCHER 

73 

91 

46 

32 

36 

26 

A 

RESEARCHER 

56 

75 

28 

42 

40 

27 

B 

COMMON 

38 

51 

26 

25 

26 

15 

OCCURRENCES 

FINAL 

84 

81 

44 

42 

57 

32 

CONCURRENCE 

PERCENT 

45 

63 

60 

60 

65 

47 

CORRELATION 

The  analysis  of  the  encoded  data  was  both  quantitative  and 
substantive.  The  quantitative  analysis  compared  the  relative 
frequency  distributions  of  the  codes  within  the  cateqories 
listed  in  Table  3.  The  substantive  analysis  studied  the 
actual  content  of  the  information  introduced  at  each  occur¬ 
rence  to  search  for  trends  that  would  not  be  detected  by 
quantitative  means. 

3.2.4  Findings 

Four  major  findings  extracted  from  analysis  of  the  proto¬ 
cols  are  summarized  below.  The  subsequent  sections  describe 
the  motivation,  the  analysis  procedure  and  the  justification 
which  lead  to  each  finding. 
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Finding  1:  Concept  generation  was  influenced  by  information 
about  system  topics  (e.g.,  distribution  environment,  produc¬ 
tion  processes,  user  environment,  corporate  environment) ,  as 
much  as  by  information  about  the  specific  design  task  (i.e., 
package  a  product) . 

Finding  2 :  The  focus  of  the  information  introduced  was 
divided  among  three  primary  categories: 

a.  key  performance  parameter  of  the  specific  design  task, 

b.  key  performance  parameter  of  the  system, 

c.  other  parameters  defining  the  system  boundaries. 
Finding  3 :  Information  acquisition  is  an  integral  part  of  the 
individual's  design  process,  with  two  common  characteristics: 

a.  Methodical  search  vs,  opportunistic  behavior:  the  design 
process  may  begin  with  a  methodical  search  for  information. 
However,  once  an  initial  concept  was  formulated,  methodical 
searching  ceased  and  opportunistic  behavior  was  observed, 
interspersing  concept  generation  and  information  seeking. 

b.  Vertical  thinking  [63,  pp.  39-45]:  pursuing  single 
concepts  and  using  information  and  knowledge  to  overcome 
obstacles  to  the  success  of  that  concept. 

Finding  4:  The  distribution  of  topics  of  information  intro¬ 
duced  during  the  design  process  correlated  to  the  distribu¬ 
tion  of  topics  identified  as  important  and  frequently  needed 
in  the  survey  of  practicing  engineers  (page  17) . 
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3. 2. 4.1  Finding  1 

Concept  generation  was  influenced  by  information  about 
system  topics  (e.g.,  distribution  environment,  production 
processes,  user  environment,  corporate  environment)  as  much 
as  by  information  about  the  specific  design  task  (i.e., 
package  a  product) .  The  motivation  to  investigate  this  topic 
came  from  the  surveys.  The  surveys  indicated  that  informa¬ 
tion  about  system  topics  was  as  important  as  information 
about  the  specific  design  task.  This  prompted  investigation 
of  whether  that  information  was  important  for  concept  genera¬ 
tion,  or  only  for  establishing  the  system  boundaries. 

The  quantitative  analysis  required  categorizing  information 
introduced  into  the  process  as  being  focused  on  either  the 
system  or  the  design  task.  Systems  information  is  that  which 
was  coded  S,D,I  or  X  (process,  distribution,  internal  or 
external  environment) .  Design  task  information  is  that  which 
was  coded  P,C  or  K  (product,  component  or  package).  The 
number  of  concepts  introduced  that  are  either  concurrent  with 
or  immediately  preceded  by  systems  information  were  then 
tabulated.  Concurrent  occurrences  of  system  information  and 
concepts  were  low,  ranging  from  10%  to  25%.  Concepts 
preceded  by  systems  information  were  high,  ranging  from  45% 
to  82%.  Thus,  the  quantitative  information  is  not  con¬ 
clusive,  but  indicates  that  systems  information  may  have  been 
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influenced  by  concept  generation  as  often  as  design  task 
information. 

For  the  substantive  evaluation,  the  context  of  each 
occurrence  where  a  concept  was  introduced  was  examined  to 
determine  what  information  may  have  influenced  its  genera¬ 
tion.  For  examples: 

Expert  A  (HDTV),  lines  10-14:  "..with  this  high  end 
product,  I  doubt  that  the  dealer  will  order  more  than  a 
couple  at  a  time  [HDGN  System].  So  with  that,  I  will 
look  to  design  packaging  for  this  thing  as  an  individual¬ 
ly  packaged  unit  [WKGN1  Task! ,  as  opposed  to  a  bulk  pack 
unit. " 

Assumed  information  about  distribution  contributed  to  the 

concept  of  individual  packaging.  Thus,  the  introduction  of 

system  information  immediately  preceding  a  concept  was 

determined  to  have  significant  influence  on  that  concept. 

Expert  C  (Pizza),  lines  54-70:  "Tomato  based  sauces  come 
in  cans  [HCGN  Task! ,  but  I  would  think  in  order  to 
possibly  keep  this  a  simple  package,  one  where  you  can 
package  all  the  ingredients  really  easily  together  would 
be  to  put  it  in  a  flexible  type  package  [HKGT1  Task! .... 
Cheese  could  be  the  same  way  [HKGT1  Task] .  This  way  the 
tomato  paste  and  the  cheese  can  be  set  on  top  of  the 
crust  in  the  package  [WKGT1  Taskl  and  take  minimum  amount 
of  space.  This  would  allow  the  sauce  and  cheese  to  be 
kept  in  the  same  general  shape  and  put  on  top  of  the 
crust,  which  would  allow  it  to  be  packaged  fairly  easily. 

If  we  go  through  the  process  now.  We  have  a  5  x  6  inch 
piece  of  pizza  crust,  the  tomato  based  sauce  in  a 
flexible  package  and  the  cheese  in  a  flexible  package. 
The  crust  can  be  put  out  on  a  continuous  machine  [HSST1 
System!  and  the  5X6  slices  cut  out  of  it  and  then  put 
on  a  piece  of  release  paper  [HKST1  System]." 

Concepts  for  packaging  the  tomato  sauce  and  cheese  in 

flexible  pouches  were  influenced  primarily  by  historical 

information  about  package  types,  a  design  task  specific 


topic.  Concepts  about  the  continuous  machine  and  release 
paper  were  influenced  by  historical  information  about 
manufacturing  processes,  a  system  topic. 

Among  the  six  protocols,  the  percentage  of  concepts 
influenced  by  system  information  ranged  from  33%  to  60%,  with 
a  mean  of  51%  and  a  standard  deviation  of  11%.  This  evidence 
supports  the  finding  that  concept  generation  was  influenced 
by  information  about  system  topics  as  much  as  by  information 
about  the  specific  design  task. 

3. 4. 1.2  Finding  2 

The  focus  of  the  information  introduced  was  divided  among 
three  primary  categories: 

a.  key  performance  parameter  of  the  specific  design  task, 

b.  key  performance  parameter  of  the  system, 

c.  other  parameters  defining  the  system  boundaries. 

The  motivation  for  this  investigation  was  to  determine  what 
portion  of  the  information  introduced  was  focused  on  the 
critical  design  issues,  and  what  portion  was  focused  on 
keeping  the  design  within  a  proper  frame  of  reference. 

The  context  of  each  coded  occurrence  was  evaluated  to 
determine  if  it  could  be  classified  into  one  of  the  three 
categories.  Percentages  shown  in  Table  5  are  based  on 
comparison  to  the  total  number  of  occurrences.  The  remaining 
occurrences,  ranging  from  16%  to  40%,  were  divided  among 
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several  topics.  For  the  HDTV  and  pizza  problems,  approxi¬ 
mately  10%  to  15%  of  the  information  focused  on  refining  the 
concept  being  developed  (e.g.,  where  to  put  graphics  on  a 
pizza  carton  or  where  to  pack  the  HDTV  remote  control) . 

In  five  of  six  protocols,  the  focus  of  information  was 
equally  distributed  among  the  three  topics,  indicating  that 
keeping  the  design  in  perspective  within  the  system  boun¬ 
daries  is  as  important  as  satisfying  performance  require¬ 
ments  . 


Table  5.  Percentage  of  Occurrences  by  Focus 


PRODUCT 

KEY  DESIGN 

KEY  SYSTEM 

DEFINING 

&EXPERT 

PARAMETER 

PARAMETER 

BOUNDARIES 

HDTV 

PRODUCT 

DISTRIBUTION 

FRAGILITY 

REQUIREMENTS 

A 

35% 

23% 

27% 

E 

21% 

16% 

23% 

PIZZA 

PRODUCT 

PACKAGING 

DEGRADATION 

PROCESS 

C 

36% 

25% 

23% 

D 

24% 

43% 

7% 

SWAB 

LEAKAGE  OR 

PACKAGING 

CONTAMINATION 

SPEED/COST 

B 

11% 

32% 

17% 

F 

22% 

38% 

16% 
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3. 2. 4. 3  Finding  3 

Information  acquisition  is  an  integral  part  of  the  in¬ 
dividual's  design  process,  with  two  common  characteristics: 

a.  Methodical  search  vs.  opportunistic  behavior 

b.  Vertical  thinking 

The  motivation  for  this  investigation  was  to  determine  if 
common  patterns  existed  among  the  six  experts  in  their 
approaches  to  seeking  information.  If  a  carmen  pattern 
existed,  it  could  be  the  foundation  for  developing  a  method 
that  could  benefit  other  designers. 

The  analysis  began  with  a  search  for  sequential  patterns 
by  comparing  coded  occurrences  for  the  total  design  effort 
and  for  quarterly  periods.  Referring  to  Table  3  (page  28) , 
the  frequency  distributions  of  focus,  level,  class  and  ter¬ 
minology  were  compared.  No  discernable  patterns  were 
discovered.  For  example,  there  was  no  evidence  of  transition 
from  general  to  specific  information,  from  nontechnical  to 
technical  terminology,  or  from  historical  data  to  requested 
data.  Information  appeared  to  be  integral  to  and  initiated 
by  events  in  each  individual's  process  of  design. 

The  next  step  of  the  analysis  was  to  determine  if  informa¬ 
tion  identification  was  deliberate  or  opportunistic.  A 
pattern  of  progression  from  deliberate  to  opportunistic 
behavior  was  observed  by  Ullman,  Stauffer  and  Dietterich  [52] 
during  protocol  analysis  of  mechanical  engineers  involved  in 
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design  of  mechanisms.  Expert  A  (HDTV),  Expert  B  (Swab)  and 
Expert  D  (Pizza)  each  began  the  design  process  with  a 
deliberate  search  for  information  about  the  system,  primarily 
focused  on  the  key  system  performance  parameter.  Expert  C 
(Pizza)  and  Expert  E  (HDTV)  began  with  a  deliberate  search 
for  information  about  the  product,  again  focused  on  the  key 
performance  parameters.  Expert  F  bypassed  any  deliberate 
questioning  and  began  organizing  thoughts  leading  toward  the 
initial  concept.  In  all  six  protocols,  once  a  general 
concept  was  identified,  no  evidence  of  deliberate  searches 
was  observed.  In  terms  of  information  identification, 
opportunistic  behavior  was  observed  in  each  experiment,  where 
deliberate  behavior  was  observed  in  five  of  the  six  proto¬ 
cols,  but  was  confined  to  the  early  stages  of  the  design 
process.  Transition  from  deliberate  searches  to  opportunis¬ 
tic  behavior  was  observed  in  five  of  the  six  protocols,  which 
is  a  pattern  consistent  with  previous  studies. 

The  third  step  in  the  analysis  was  to  determine  if  in¬ 
dividual  experts  applied  vertical  thinking,  as  evidenced  by 
the  pursuit  of  single  concepts.  Vertical  thinking  was  also 
reported  by  Ullman,  Stauffer  and  Dietterich  [52,  p.  13]  in 
previous  protocol  analysis  experiments.  Four  of  the  six 
experts  pursued  single  concepts,  incorporating  information 
as  it  was  required  to  overcome  barriers.  Two  experts,  both 
addressing  the  cotton  swab,  introduced  more  than  one  concept 


35 


for  portions  of  their  design.  Expert  B  was  deliberate  in 
writing  down  alternative  approaches  to  overcome  the  two  key 
performance  requirements  (i.e.,  no  leakage  and  high  produc¬ 
tion  speed) .  This  led  to  concepts  of  both  flexible  plastic 
and  rigid  glass  packages.  Once  the  glass  concept  was 
rejected,  it  was  not  reintroduced.  Expert  F  pursued  dif¬ 
ferent  production  concepts  (e.g.,  molded,  blister  pouch,  or 
form,  fill  and  seal)  for  the  single  package  concept.  In 
general,  the  experts  followed  a  single  concept  through  the 
entire  design  process,  demonstrating  a  pattern  of  vertical 
thinking. 

Based  on  these  three  analytical  steps,  it  is  concluded  that 
the  acquisition  of  information  cannot  b-_  naturally  separated 
from  the  individual  design  process.  No  common  approach  over 
time  was  observed,  but  evidence  of  a  transition  from  methodi¬ 
cal  searching  to  opportunistic  behavior  and  vertical  thinking 
was  observed. 

3. 2. 4. 4  Finding  4 

The  distribution  of  topics  of  information  introduced  during 
the  design  process  correlated  to  the  distribution  of  topics 
identified  as  important  and  frequently  needed  in  the  survey 
of  practicing  engineers  (page  17) . 

The  motivation  for  this  investigation  was  to  compare  data 
from  two  distinct  sources  and  two  sample  populations.  Survey 
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data  is  retrospective  and  subject  to  filtering  by  the 
respondent.  Protocol  analysis  data  captures  a  portion  of  the 
participant's  thoughts  at  approximately  the  time  they  occur. 
Correlation  of  the  two  types  and  sources  of  data  concerning 
the  same  question  would  increase  credibility  of  the  con¬ 
clusions  reached. 

The  relative  frequency  of  the  topics  for  which  information 
was  introduced  is  shown  in  Table  6.  Component  and  product 
categories  were  combined  to  relate  to  the  survey  categories. 
The  zero  entries  for  distribution  in  both  protocols  addres¬ 
sing  the  cotton  swab  were  further  investigated.  As  discussed 
in  section  3.2.3,  some  occurrences  could  have  been  coded  more 
than  one  way,  especially  the  focus  code.  The  researcher  must 
evaluate  the  primary  focus  of  the  information.  For  the 
cotton  swab,  occurrences  discussing  the  number  of  swabs  per 
carton  could  have  been  coded  either  as  a  package  or  a 
distribution  focus.  Because  of  the  context  in  which  they 
were  entered,  those  occurrences  appeared  focused  on  the 
package.  Secondly,  cotton  swabs  are  relatively  insensitive 
to  the  distribution  environment.  During  these  short  duration 
experiments,  the  experts  appeared  to  direct  their  efforts 
toward  the  critical  issues.  Consequently,  secondary  issues 
may  not  have  received  the  attention  that  would  have  been 
present  in  an  actual  design  in  a  corporate  environment. 
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Table  6.  Relative  Frequency  of  Information  Topics 


CATEGORY 

HDTV 

A  E 

PIZZA 

C  D 

SWAB 

B  F 

AVERAGE 

PRODUCT 

28 

26 

25 

15 

35 

9 

23.0 

PACKAGING 

27 

39 

41 

55 

22 

50 

39.0 

PROCESS 

11 

5 

14 

21 

20 

9 

13.3 

DISTRIBUTION 

24 

14 

11 

2 

0 

0 

8.5 

INTERNAL 

6 

7 

0 

5 

9 

19 

7.7 

EXTERNAL 

5 

9 

9 

2 

15 

13 

8.3 

Table  7  displays  the  top  categories  identified  in  two 
survey  questions  and  in  the  protocols.  Question  20,  Survey 
1,  asked  what  areas  of  information  were  needed.  Question 
6,  Survey  2,  asked  which  topics  were  important.  The  ter¬ 
minology  is  different  between  the  survey  questions  and  the 
categories  used  during  the  protocols.  For  example,  the 
protocol  category  "packaging"  included  both  packaging 
materials  and  package  features  (e.g.,  zip-open  carton  for 
pizza,  peel-top  package  for  the  cotton  swab) .  Marketing 
requirements  from  the  surveys  would  be  divided  among  the 
categories  of  distribution,  internal  environment,  and 
external  environment  in  the  protocols.  Budget  information, 
identified  in  question  6  of  the  follow-up  survey,  would  be 
considered  internal  environment  during  the  protocols. 


Allowing  for  differences  m  terminology  used  in  the  surveys 
and  the  protocol,  the  lists  are  similar  but  not  identical. 
The  coding  used  on  the  protocols,  the  general  and  retrospec¬ 
tive  nature  of  the  survey  in  contrast  to  the  specific  design 
problems  of  the  protocols,  and  the  suggested  topics  in  the 
survey  questions  may  have  contributed  to  the  difference. 
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4  A  PROBLEM  FORMULATION  METHOD 

Based  on  the  preliminary  research,  it  is  hypothesized  that 
deliberate  identification  of  system  boundaries  and  the  key 
performance  requirements  of  both  the  specific  design  task  and 
the  system  parameters  should  occur  during  problem  formulation 
and  prior  to  concept  generation.  From  the  protocol  analysis, 
the  methodical  search  for  information  at  the  outset  of  the 
design  process  may  be  viewed  as  an  attempt  to  "bring  manage¬ 
ability  and  closure  to  the  problem"  [3]  at  the  earliest 
possible  stage  by  defining  the  conditions  under  which  any 
solution  can  be  viewed  as  complete.  It  is  important  to  note 
that  in  establishing  the  system  boundaries  and  the  required 
levels  of  performance,  the  experts  did  not  limit  their 
ability  to  creatively  develop  concepts.  They  only  defined 
the  region  within  which  to  be  creative,  with  hope  of  avoiding 
ineffectiveness  and  Errors  of  the  Third  Kind  (i.e.,  solving 
the  wrong  problem  or  a  "suboptimal"  problem)  [64].  MacCrim- 
mon  and  Taylor  [65]  identify  "determining  the  boundaries  of 
a  problem"  as  one  of  four  strategies  for  reducing  the  com¬ 
plexities  of  problem  formulation. 

From  both  the  surveys  and  the  protocols,  the  consistency 
of  categories  for  which  information  is  required  indicates 
that  identification  of  key  performance  parameters  about  both 
the  design  task  at  hand  and  the  environment  within  which  the 
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task  must  be  solved  are  important  to  the  process.  The 
surveys  provided  no  evidence  that  technical  performance 
requirements  were  a  deliberate  part  of  the  design  process. 
The  protocols  provided  evidence  that  the  experts  introduced 
technical  performance  requirements  early  in  the  design 
process,  but  stopped  short  of  deliberately  and  explicitly 
stating  them. 

Therefore,  the  second  phase  of  the  research  hypothesized 
and  validated  the  effect  of  using  a  method  which  incorporates 
technical  performance  requirements  on  the  quality  of  problem 
formulation  in  engineering  systems  design.  A  problem 
formulation  method  was  developed,  criteria  and  a  model  for 
evaluating  the  effect  of  the  method  were  defined,  and  the 
method  was  experimentally  validated. 

Two  goals  guided  the  development  of  a  problem  formulation 
method.  First,  the  method  should  assist  in  the  identifica¬ 
tion  of  system  boundaries  and  the  key  performance  criteria 
of  both  the  specific  design  task  and  the  system  parameters. 
Secondly,  the  method  should  encourage  the  use  of  technical 
performance  requirements  as  a  means  of  defining  those 


boundaries  and  criteria. 
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4 . 1  Method  Development 

To  satisfy  the  first  goal,  Dimensional  Analysis  was  applied 
to  engineering  systems  design.  Dimensional  Analysis  is  an 
analytical  technique  designed  to  clarify  and  explore  the 
dimensions  and  limits  of  general  problems.  Dimensional 
Analysis  recognizes  the  five  aspects  of  general  problems  as 
substantive,  spatial,  temporal,  quantitative  and  qualitative 
[66,  pp.  60-66].  Relating  data  from  the  surveys,  five 
characteristic  aspects  of  engineering  systems  problems  were 
identified: 

a.  time  (e.g.,  dates,  duration,  cycles,  etc.) 

b.  dimensions  (e.g.,  distance,  areas,  weights,  etc.) 

c.  value  (e.g. ,  precision,  cost,  quality,  reliability, 
etc. ) 

d.  physical  matter  (e.g.,  materials,  parts,  machines,  etc.) 

e.  people  (e.g.,  user,  manufacturer,  organizations,  etc.) 

The  five  characteristic  dimensions  are  not  purported  to  be 

unique,  universal,  or  inclusive.  However,  they  are  con¬ 
sidered  to  be  sufficiently  representative  and  general  to 
assist  in  the  identification  of  system  boundaries  and  the  key 
performance  criteria  of  both  the  specific  design  task  and  the 
system  parameters. 
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To  satisfy  the  second  goal  of  method  development,  a  concise 
description  of  technical  performance  requirements  was 
formulated.  Technical  performance  requirements  describe  in 
measurable  or  verifiable  terms  the  levels  of  performance 
required  for  any  solution  to  be  considered  acceptable  and 
complete.  Each  technical  performance  requirement  should 
include  four  elements: 

Identification  -  a  verbal  description  of  the  performance 
attribute  (e.g. ,  strength,  speed,  power,  durability,  weight) 
Conditions  -  delineation  of  the  environmental  conditions 
under  which  the  performance  is  required  (e.g.,  temperature, 
moisture  content,  time,  number  of  operators) 

Target  Value  -  the  optimum  performance  value,  that  value 
which  will  minimize  loss  or  maximize  gain  (e.g.,  32  miles 
per  gallon,  90  decibels,  63  hertz,  $1200) 

Allowable  Variance  -  the  permissible  deviation  from  the 
target  value  (e.g.,  ±  2%,  no  more  than,  +  6  hertz) 

Concisely  stated,  the  problem  formulation  method  uses  five 
dimensions  to  assist  in  identifying  boundary  and  performance 
criteria,  and  technical  performance  requirements  as  a  means 
of  expressing  them.  The  method  can  be  conveyed  using  a 
guide,  as  shown  in  Figure  5. 
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Guide  to  Problem  Formulation 


During  problem  formulation,  you  should  identify  two  elements:  the 
boundaries  of  the  problem  and  the  key  performance  criteria. 
Describing  these  two  elements  will  allow  you  to  focus  your  creative 
efforts  within  the  proper  boundaries,  and  determine  when  your  design 
is  sufficient.  Writing  technical  performance  requirements  will  help 
you  describe  these  elements.  Technical  performance  requirements 
describe  in  measurable  or  verifiable  terms  the  performance  required 
for  any  solution  to  be  considered  acceptable.  Each  technical 
performance  requirement  should  include  four  elements: 

Identification  -  a  verbal  description  of  the  performance  attribute 
(e.g.,  strength,  speed,  cost,  durability,  weight) 

Conditions  -  delineation  of  the  environmental  conditions  under  which 
the  performance  is  required  (e.g.,  temperature,  moisture  content, 
time,  number  of  operators) 

Target  Value  -  the  optimum  performance  value,  that  value  which  will 
minimize  loss  or  maximize  gain  (e.g.,  32  miles  per  gallon,  90 

decibels,  63  Hertz) 

Allowable  Variance  -  the  permissible  deviation  from  the  target  value 
(e.g.,  +  2%,  no  more  than,  +  6  Hertz) 

It  is  important  that  you  describe  WHAT  performance  you  need,  but 
not  how  to  achieve  it.  You  should  not  restrict  the  designer's 
creative  ability,  but  should  instead  identify  the  boundaries  within 
which  to  be  creative.  If  you  specify  what  levels  of  performance  are 
required  for  any  solution,  the  designer  should  be  able  to  determine 
if  a  solution  is  within  those  boundaries  and  also  when  the  solution 
is  adequate  enough  to  stop  designing. 

To  help  identify  performance  attributes,  think  and  ask  questions 
in  terms  of: 

TIME  (dates,  duration,  cycles,  etc.) 

DIMENSIONS  (distance,  areas,  weights,  etc.) 

VALUE  (precision,  cost,  quality,  reliability,  etc.) 

PHYSICAL  MATTER  (materials,  parts,  machines,  etc.) 

PEOPLE  (user,  manufacturer,  organizations,  etc.) 

Use  the  answers  to  these  questions  to  write  as  many  technical 
performance  requirements  as  you  feel  are  necessary  to  adequately 
formulate  the  problem. 


Figure  5.  Problem  Formulation  Guide 
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4.2  Evaluation  Criteria  and  Model 

Evaluation  criteria  and  an  evaluation  model  were  developed 
in  order  to  validate  the  effectiveness  of  the  problem 
formulation  method.  A  Multiple  Attribute  Value  model  was 
selected  as  an  appropriate  evaluation  tool. 

Multiple  Attribute  Value  models,  also  known  as  Weighted 
Criterion  Function  or  Multiattribute  Utility  Function  models, 
are  appropriate  for  addressing  value  trade-offs  among 
multiple  objectives  and  uncertainty  in  complex  problems  [67]. 
The  attributes  should  be  independent  of  each  other  and  each 
has  its  own  utility  scale.  In  addition,  a  relative  weight 
is  assigned  to  each  attribute.  For  a  given  option  (e.g., 
problem  statement)  a  value  may  be  subjectively  assigned  for 
each  attribute.  The  weighted  sum  of  values  defines  the 
utility  of  the  option.  In  uncertainty  situations,  the  option 
with  the  highest  expected  utility  should  be  chosen.  Problem 
solving  can  be  viewed  as  an  uncertainty  situation  in  that  the 
highest  quality  problem  statement  will  not  ensure  the  highest 
quality  solution. 

Sakman  [22]  used  a  panel  of  experts  to  develop  a  Multiple 
Attribute  Value  model  to  evaluate  the  quality  of  problem 
statements.  Previous  work  had  used  the  quality  of  solutions 
to  indirectly  indicate  the  quality  of  problem  statements 
[3,68].  Sakman* s  five  attributes  and  utility  scales  of 
quality  for  problem  statements  are: 
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XO:  Clarity  -  The  binary  assessment  of  whether  or  not  the 
problem  is  stated  in  a  coherent  way  and  in  understandable, 
nonjargon  terms  sufficient  to  allow  further  evaluation  of 
attributes . * 


0  l 
NOT-CLEAR  CLEAR 
(Unacceptable)  (Acceptable) 

XI:  Solution  Implication  -  The  degree  to  which  the 
statement  suggests  solutions  or  ways  to  achieve  solutions  to 
the  problem.  The  more  a  statement  does  so,  the  less  favora¬ 
ble  it  is  (i.e.,  this  dimension  has  a  negative  orientation). 


0 _ 

SOLVES 

(Worst) 


LEADS 

(Anchor) 


_ 100 

OPEN-ENDED 

(Best) 


Clarity  was  originally  intended  as  a  relative  measure, 
but  was  ultimately  used  to  eliminate  problem  statements  which 
lacked  sufficient  clarity  to  permit  evaluation  of  the 
remaining  four  attributes. 


46 


X2:  Structural  Considerations  -  The  degree  to  which  the 
statement  reflects  consideration  for  all  relevant  and  unique 
aspects,  parts,  complexities  and  their  interrelationships  and 
intrarelationships  to  the  extent  that  can  be  seen  or  verified 
at  the  stage  of  problem  formulation. 


0 _ 100 

NONE  SOME  ALL 

(Worst)  (Anchor)  (Best) 

X3 :  Stakeholders  -  The  degree  to  which  the  statement 
reflects  considerations  of  the  needs,  interests  and  power  of 
all  stakeholders,  i.e.,  parties  that  will  affect  or  be 

affected,  directly  or  indirectly,  by  the  solution  of  the 
problem.  Stakeholders  may  be  tangible  (e.g. ,  an  interest 
group)  or  intangible  (e.g.,  the  "unborn"  generation),  may 

have  high  salience  (in  the  sense  of  being  motivated,  having 
high  stake)  or  low  salience,  and  may  be  vocal/outspoken  or 
"invisible."  (Note:  It  is  assumed  that  all  salient  stake¬ 
holders  are  tangible,  but  not  all  tangible  stakeholders  are 
salient. ) 


0 _ 

NONE 

(Worst) 


SALIENT 
(Anchor  #1) 


TANGIBLE 
(Anchor  #2) 


100 

ALL 

(Best) 
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X4:  Level  of  Focus  -  The  degree  to  which  the  statement 
defines  as  high  a  level  or  as  broad  a  problem  as  is  possible 
and  appropriate. 

0 _ 100 

NARROW  SOMEWHAT  SOMEWHAT  BROAD 

NARROW  BROAD 

(Worst)  (Anchor  #1)  (Anchor  #2)  (Best) 

Figure  6  illustrates  the  Multiple  Attribute  Value  model 
used  by  Sakman  to  evaluate  problem  statements  developed  by 
small  groups  for  general  problems  [22]  and  by  Nadler  to 
evaluate  problem  statements  developed  by  individuals  solving 
mechanical  engineering  problems  [19].  Both  research  efforts 
compared  the  problem  statements  developed  using  problem 
definition  methods  selected  from  those  available  in  the 
literature.  In  both  research  efforts,  significant  differen¬ 
ces  between  methods  were  detected  for  individual  attributes, 
but  no  significant  difference  was  detected  for  overall 
problem  statement  quality. 
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ATTRIBUTE 

WEIGHT 

UTILITY  SCALE 

SCORE 

UTILITY 

CLARITY 

1 

0  1 

SO 

SO 

SOLUTION 

IMPLICATION 

Wl= . 28 

0  100 

SI 

W1XS1 

STRUCTURAL 

CONSIDERATION 

m 

0  100 

S2 

W2xS2 

STAKEHOLDERS/ 
INTEREST  GRPS 

W3= . 27 

0  100 

S3 

W3XS3 

LEVEL  OF 

FOCUS 

£ 

.fe¬ 

ll 

BHB 

S4 

W4XS4 

QUALITY= 

SO  *  £(Wi  *  Si) 
i=l ,2,3,4 

Figure  6.  Sakman's  Multiple  Attribute  Value  Model 


Sakman's  model  was  developed  for  general  problems,  and 
tested  on  both  nonengineering  and  engineering  problems.  To 
reflect  the  importance  of  identifying  and  defining  boundaries 
and  performance  criteria,  two  original  attributes  are  intro¬ 
duced: 

Performance  Identification  (X5)  -  The  degree  to  which  the 
statement  defines  what  performance  is  reguired  to  satisfy  the 
identified  needs  of  the  problem. 


0 _ 

NOT  DEFINED 
(Worst) 


SOMEWHAT  DEFINED 
(Anchor) 


100 

DEFINED 

(Best) 


Performance  Measurability  (X6)  -  The  degree  to  which  the 
required  performance  can  be  measured  or  verified. 

0 _ 100 

NOT  MEASURABLE  SOMEWHAT  MEASURABLE  MEASURABLE 

(Worst)  (Anchor)  (Best) 

Structural  Considerations  (X2)  and  Stakeholders  (X3) 
measure  the  degree  to  which  the  problem  statement  reflects 
consideration  for  aspects,  parts,  and  complexities  of  the 
problem  and  for  needs,  interest  and  power  of  stakeholders. 
Performance  Identification  and  Performance  Measurability 
address  similar  aspects,  parts,  complexities  and  stakehold¬ 
ers,  but  from  a  more  appropriate  perspective.  The  purpose 
of  engineering  design  is  to  develop  systems,  components  or 
processes  that  perform  in  such  a  manner  as  to  satisfy  needs. 
Therefore  it  is  more  appropriate  to  identify  and  measure  the 
performance  required  by  a  part  or  a  stakeholder  than  it  is 
to  identify  the  existence  of  the  part  or  stakeholder. 

4 . 3  The  Experiment 

A  controlled  task,  guided  response  experiment  was  used  to 
evaluate  the  effect  of  the  method  on  the  quality  of  problem 
formulation.  The  test  subjects  were  practicing  mechanical 
engineers  and  graduate  students  in  mechanical  engineering. 
Two  independent  graders  evaluated  each  statement  for  clarity 
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(X0)  and  along  each  of  the  six  dimensions  (XI  though  X6) 
described  in  Section  4.2. 

4.3.1  Experimental  Task 

General  instructions  to  the  test  subjects  are  Appendix  D.l. 
Each  participant  was  provided  the  problem  description  shown 
in  Appendix  D.2  and  tasked  to  formulate  the  problem.  Half 
of  the  test  subjects,  selected  at  random,  were  asked  to  apply 
the  problem  formulation  method  by  following  the  guidelines 
in  Appendix  D.3.  The  test  administrator  used  the  information 
in  Appendix  D.4  to  answer  specific  questions  from  individual 
test  subjects,  as  opposed  to  being  made  openly  available  to 
every  test  subject.  All  interaction  between  the  test  subject 
and  the  test  administrator  was  done  with  handwritten  ques¬ 
tions  and  responses. 

4.3.2  Experimental  Design 

The  experiment  was  conducted  as  a  23  factorial  design, 
signifying  three  factors  or  treatments,  each  at  two  levels. 
One  factor  was  the  approach  used,  either  unstructured 
(control  group)  or  the  problem  formulation  method.  The 
second  factor  was  the  classification  of  participants  as 
either  practicing  engineers  or  graduate  students.  The  third 
factor  was  the  grader.  The  hypotheses  evaluated  the  effect 
each  of  the  three  treatments  on  the  attributes  of  quality  (XI 
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through  X6)  and  on  the  weighted  aggregate  value  of  quality. 
Clarity  (XO)  was  used  only  to  eliminate  problem  statements 
which  lacked  sufficient  clarity  for  further  evaluation.  The 
Multiple  Attribute  Value  model  used  for  evaluation  is 
Appendix  D.5.  Instructions  to  the  graders  are  shown  in 
Appendix  D.6. 

4.3.3  Sample  Population 

The  sample  population  consisted  of  eight  mechanical 
engineers  and  eight  graduate  students  in  mechanical  en¬ 
gineering.  Of  the  sixteen  participants,  two  had  earned 
doctoral  degrees.  Seven  of  the  sixteen  participants  had 
earned  Master  of  Science  in  Mechanical  Engineering  degrees, 
of  whom  four  were  studying  towards  doctoral  degrees.  The 
remaining  seven  participants  had  earned  Bachelor  of  Science 
in  Mechanical  Engineering  degrees,  of  whom  three  were 
studying  toward  advanced  degrees. 

The  practicing  engineers  were  currently  employed  as  product 
design  engineers,  senior  design  engineers,  or  design  and 
analysis  engineers.  Industry  experience  ranged  from  8  months 
to  24  years.  Participation  in  the  experiment  was  voluntary, 
and  both  individual  and  corporate  anonymity  was  assured.  The 
engineers  worked  for  one  of  two  corporations  in  the  Houston, 
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Texas,  metropolitan  area.  The  corporations  differed  in  their 
product  and  service  areas,  corporate  size,  and  geographic 
span. 

The  graduate  students  were  all  enrolled  at  Texas  A&M 
University  in  College  Station,  Texas.  Each  of  the  graduate 
students  had  received  an  undergraduate  degree  from  an 
institution  other  than  Texas  A&M  University.  The  four 
students  pursuing  their  doctoral  degrees  received  their 
master  degrees  from  Texas  A&M  University.  Four  of  the 
graduates  students  also  had  from  1  to  4  years  of  industry 
experience  as  mechanical  engineers. 

4.3.4  Graders 

The  independent  graders  were  selected  as  representative  of 
academic  and  industry  perspectives  of  engineering  systems 
design.  Grader  S  is  an  associate  professor  of  mechanical 
engineering  in  an  ABET-accredited  program.  Grader  S  earned 
a  Doctor  of  Philosophy  degree  in  mechanical  engineering  and 
is  a  registered  Professional  Engineer.  Grader  S  has  taught 
courses  covering  the  engineering  design  process,  to  include 
problem  formulation  and  Multiple  Attribute  Value  models. 

Grader  J  currently  directs  the  operational  testing  program 
for  a  military  hardware  system,  verifying  that  the  equipment 
and  all  supporting  systems  meet  the  user  performance  require¬ 
ments  prior  to  full-scale  production.  Grader  J  has  over  10 
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years  experience  in  research,  development  and  testing  of 
military  hardware  systems.  Grader  J  earned  a  Master  of 
Science  degree  in  Operations  Research. 

4.4  Analysis  Procedure 

The  handwritten  problem  statements  were  typed  verbatim  onto 
grading  forms  for  consistent  appearance  (Appendix  E.l).  Each 
problem  statement  was  evaluated  by  two  independent  graders, 
neither  of  whom  were  familiar  with  the  experimental  design 
or  the  focus  of  the  research  project.  Results  of  the 
evaluations,  coded  for  analysis,  are  shown  in  Appendix  E.2. 
Analysis  of  variance  (ANOVA)  was  used  to  determine  the  effect 
of  the  three  factors  (method,  eng ineer/ student ,  and  grader) 
on  the  quality  of  the  problem  statement. 

Analysis  of  variance  is  a  procedure  in  which  the  total 
variation  in  a  measured  response  is  partitioned  into  com¬ 
ponents  which  can  be  attributed  to  identifiable  sources  of 
variation,  called  factors  or  treatments  [69].  The  objective 
of  ANOVA  is  to  make  statistical  inferences  about  the  means 
of  independent,  random  samples  drawn  from  two  or  more  popula¬ 
tions.  The  null  hypothesis  to  be  tested  is  that  all  of  the 
population  means  (Mi)  are  equal,  i.e., 

H0:  Mi  =  M2  =  •  •  •  =  Mk  for  K  treatments 
Hx:  Mi  t  Mj  for  some  i  and  j.. 

The  fundamental  test  is  the  comparison  of  the  variance 
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between  treatments  (SSTr,  treatment  sum  of  squares)  to  the 
variance  within  treatments  (SSE,  residual  or  error  sum  of 
squares)  by  using  an  appropriate  test  statistic  based  on  the 
F  Distribution.  The  test  statistic  is  the  ratio  of  treatment 
mean  square  to  error  mean  square,  or  MSTr/MSE,  where 

MSTr  =  SSTr/(k-l)  for  k  treatments 

MSe  =  SSE/ (N-k)  for  a  sample  population  of  size  N. 

If  the  null  hypothesis  is  true,  then  the  test  statistic  is 
expected  to  be  close  to  1;  otherwise  it  is  expected  to  be 
greater  than  one.  If  the  null  hypothesis  is  true,  then  the 
test  statistic  will  follow  an  F  Distribution  with  (k-1)  and 
(N-l )  degrees  of  freedom.  The  mechanics  of  ANOVA  require  the 
calculation  of  the  appropriate  test  statistic  from  the 
experimental  data,  then  comparison  of  that  test  statistic  to 
the  value  in  an  F  Distribution  table  for  the  appropriate 
degrees  of  freedom  and  appropriate  limit  for  the  probability 
of  a  Type  I  or  Alpha  (a)  error.  If  the  test  statistic  is 
larger  than  the  F  Distribution  value,  then  the  null  hypoth¬ 
esis  is  rejected,  with  a  probability  that  the  treatment  means 
were  actually  equal.  Using  SAS  statistical  analysis  computer 
software,  the  ANOVA  procedure  calculates  the  probability  of 
obtaining  a  test  statistic  greater  than  the  value  calculated 
if  H0  is  true.  If  that  value  is  less  than  an  allowable 
significance  level,  then  the  null  hypothesis  is  rejected. 
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Hypotheses  were  tested  at  an  a=10%  significance  level,  as 
was  done  by  Sakman  [22]  and  Nadler  [19]  in  similar  experi¬ 
ments.  Ten  percent  is  appropriate  based  on  the  amount  of 
subjective  evaluation  required,  the  inability  to  identify  and 
control  all  possible  variables  and  the  lack  of  preceding 
research  [70].  At  this  level  of  significance,  differences 
in  the  quality  of  the  problem  statements  can  be  detected  with 
90%  confidence  that  the  difference  actually  exists  and  is  not 
due  to  random  error. 

4.5  Results 

The  initial  screening  of  the  problem  statements  was  to 
determine  if  they  were  sufficiently  clear  for  further 
evaluation.  Grader  J  determined  that  sample  number  24  was 
unclear  in  accordance  with  attribute  scale  X0.  Grader  S 
evaluated  number  24  but  noted  in  the  margin  that  it  was 
almost  eliminated  as  unclear.  Therefore,  sample  number  24 
was  removed  from  the  data  set. 

4.5.1  Validation  of  Assumptions 

The  assumptions  underlying  analysis  of  variance  are: 

1.  The  k  samples  (corresponding  to  k  treatments)  represent 
independent  samples  drawn  from  k  specific  populations  with 
unknown  means  Mi  =  M2  =  •••-  Mk  , 

2.  Each  of  the  k  populations  is  normally  distributed,  and 
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3.  Each  of  the  k  populations  have  common  variance  [69,  p. 
456]  . 

Independence  of  samples  is  assured  by  the  experimental 
design.  Each  sample  is  assignable  to  an  experimental  unit, 
which  is  a  unigue  combination  of  grader  (GDR) ,  control  group 
or  use  of  the  problem  formulation  method  (PFM)  ,  and  clas¬ 
sification  as  a  practicing  engineer  or  graduate  student 
(WORK) .  No  sample  can  belong  to  more  than  one  experimental 
unit. 

Normal  distribution  of  the  samples  was  verified  using 
normal  probability  plots  of  the  observed  values.  Hypothesis 
testing  based  on  the  F  Distribution,  as  used  in  analysis  of 
variance,  is  robust  to  moderate  lack  of  normality  [71,  p.  86, 
72,  p.  737].  As  shown  in  Appendix  E.3,  the  populations  plot 
along  a  generally  straight  line,  indicating  that  the  samples 
generally  follow  a  normal  distribution. 

The  assumption  of  equal  variances  between  treatments  was 
tested  based  on  the  F  Distribution.  For  a  =  0.10,  the  F 
Distribution  values  for  the  confidence  interval  of  a  two- 
tailed  test  are  f0.05,i4,i4=  2.48  and  f0.9s,i4,i4=  0.4  02.  As  shown  in 
Table  8,  three  of  the  test  statistics  are  outside  of  the 
confidence  interval.  For  comparing  means  with  unequal 
variances,  the  Smith-Satterthwaite  approximation  procedure 
was  used  [69,  p.  307-308]  and  is  annotated  in  appropriate 
tables  in  subsequent  discussion. 
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Table  8.  Test  Statistics  for  Treatment  Variances 


XI 

X2 

X3 

X4 

X5 

X6 

AGG 

GDR 

0.41 

0.58 

0.99 

0.91 

1.29 

0.77 

0.74 

PFM 

1.83 

0.19* 

0.74 

3.71* 

0.42 

2.78* 

0.61 

WORK 

0.54 

1.60 

2.05 

0.47 

1.17 

1.58 

1.40 

*  Indicates  unequal  variances  at  a  =  0.10 

4.5.2  Testing  Main  Effects  and  Interactions 

The  effect  of  the  three  primary  factors  and  all  interac¬ 
tions  was  tested  first.  As  shown  in  Appendix  E.4,  the  three- 
factor  and  two-factor  interactions  had  no  significant  effect 
on  the  quality  of  the  problem  statement. 

The  effect  of  the  three  primary  factors  on  problem  state¬ 
ment  quality  using  the  unweighted  aggregate  value  was 
investigated.  As  shown  in  Table  9,  the  use  of  the  method 
(PFM)  and  the  grader  (GDR)  had  a  significant  effect  on  the 
quality  of  the  problem  statement.  Whether  the  subject  was 
a  practicing  engineer  or  student  (WORK)  had  no  significant 
effect  on  the  quality,  eliminating  that  factor  from  subse¬ 
quent  models. 


Table  9.  Three-Factor  ANOVA 


Source 

Degrees  of 
Freedom 

F  Value 

Probability  of 
a  Value  >  F 

PFM 

1 

5.33 

0.03  * 

GDR 

1 

16.22 

<0.01  * 

WORK 

1 

0.60 

0.44 

*  Significant  at  a.  =  0.10 


The  two-factor  model  with  interaction  shown  in  Table  10 
indicated  that  there  was  no  significant  interaction  between 
PFM  and  GDR.  Therefore,  all  subsequent  models  evaluated  only 
the  effect  of  the  two  primary  factors,  PFM  and  GDR.  The 
following  sections  will  discuss  the  effect  of  each  factor  in 
detail . 


Table  10.  Two-Factor  ANOVA  with  Interaction 


Source 

Degrees  of 
Freedom 

F  Value 

Probability  of 
a  Value  >  F 

PFM 

1 

16.55 

<0.01  * 

GDR 

1 

5.44 

0.03  * 

PFM*GDR 

1 

1.15 

0.29 

*  Significant  at  a  =  0.10 
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4.5.3  The  Effect  of  the  Problem  Formulation  Method 

The  effect  of  using  the  problem  formulation  method  (PFM) 
during  problem  formulation  was  investigated  three  ways: 

a.  The  effect  of  PFM  on  each  of  the  six  attributes  of 
problem  statement  quality, 

b.  The  effect  of  PFM  on  problem  statement  quality  as 
determined  by  the  six-dimensional  Multiple  Attribute  Value 
model  as  described  in  Appendix  D.5,  and 

c.  The  effect  on  problem  statement  quality  as  determined 
by  Sakman's  four-dimensional  Multiple  Attribute  Value  model. 

4. 5. 3.1  The  Effect  of  PFM  on  Attributes  of  Quality 

As  shown  in  Tab1 e  11,  test  subjects  who  applied  the  problem 
formulation  method  were  determined  to  have  performed  signifi¬ 
cantly  better  than  test  subjects  in  the  control  group  with 
respect  to  four  of  the  six  attributes.  The  four  attributes 
are  Structural  Considerations,  Stakeholders,  Performance 
Identification,  and  Performance  Measurability.  No  sig¬ 
nificant  difference  was  determined  for  the  other  two  at¬ 
tributes,  Solution  Implication  and  Level  of  Focus.  Figure 
7  illustrates  the  differences  in  mean  scores  between  the 
control  group  (CTL)  and  the  treatment  group  (PFM)  for  each 
of  the  six  attributes. 


*  Significant  at  a  =  0.10 


**  Calculated  with  unequal  variances 
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4. 5. 3. 2  The  Effect  of  PFM  on  Aggregate  Values  of  Quality 

Test  subjects  who  applied  the  problem  formulation  method 
were  determined  to  have  performed  significantly  better  than 
test  subjects  in  the  control  group  when  comparing  weighted 
aggregate  scores  using  the  six-attribute  model.  To  test 
sensitivity  to  the  weighting  factors,  five  aggregate  models 
were  evaluated.  As  shown  in  Table  12,  an  unweighted  ag¬ 
gregate  and  four  weighted  aggregate  models  were  compared. 
The  weighting  factors  were  based  on  Sakman's  four-attribute 
model,  which  was  derived  from  a  panel  of  experts.  The  two 
additional  factors  (Performance  Identification  and  Perfor¬ 
mance  Measurability)  were  assigned  weights  ranging  from  the 
lowest  to  the  highest  value  of  the  original  four.  The  six- 
factor  model  was  then  normalized. 
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Table  12.  Sensitivity  of  Aggregate  Model  to  Weighting 


ATTRIBUTES 

WEIGHTING  FACTORS 

AGG1  AGG2  AGG3  AGG4  AGG5 

SOLUTION 

IMPLICATION 

.  167 

.219 

.187 

.177 

.  173 

STRUCTURAL 

CONSIDERATIONS 

.167 

.  242 

.207 

.  196 

.  191 

STAKEHOLDERS 

.167 

.211 

.  180 

.  171 

.167 

LEVEL  OF  FOCUS 

.  167 

.109 

.  093 

.089 

.086 

PERFORMANCE 

IDENTIFICATION 

.  167 

.109 

.  167 

.184 

.  191 

PERFORMANCE 

MEASURABILITY 

.  167 

.  109 

.167 

.  184 

.191 

F  VALUE 

16.46 

15.34 

17.73 

18.31 

18.55 

PROBABILITY  OF 
A  VALUE  >F 

<0.01 

<0.01 

<0.01 

<0.01 

<0.01 

4. 5. 3. 3  The  Effect  of  PFM  With  the  Four-Attribute  Model 

Test  subjects  who  applied  the  problem  formulation  method 
were  determined  to  have  performed  significantly  better  than 
test  subjects  in  the  control  group  when  applying  Sakman's 
four-attribute  model.  The  mean  score  for  test  subjects  using 
the  problem  formulation  method  was  62.2  out  of  100  possible 
points.  The  mean  score  for  the  test  subjects  in  the  control 
group  was  45.8  points.  The  test  statistic  value  of  10.3 
corresponds  to  a  probability  of  less  than  0.01  that  the 


difference  in  means  is  due  to  random  error.  It  is  therefore 
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concluded  with  greater  than  90%  confidence  that  the  use  of 
the  problem  formulation  method  resulted  in  higher  problem 
statement  quality. 

4.5.4  The  Effect  of  Graders 

Evaluating  the  problem  statements  was  subjective  within  the 
guidelines  provided  in  Appendix  D.6.  Consequently,  each 
grader  introduces  a  unique  set  of  personal  values  into  the 
evaluation  process.  The  values  may  stem  from  experience, 
education  or  other  uncontrolled  sources  which  helped  deter¬ 
mine  the  grader's  personal  rating  scheme.  The  mean  scores 
awarded  by  each  grader  are  shown  in  Table  13. 

Figure  8  shows  the  mean  values  assigned  by  each  grader  for 
each  of  the  six  attributes.  In  general,  Grader  J  awarded 
lower  scores  than  Grader  S.  Despite  the  numerical  differen¬ 
ces  between  scores,  the  conclusions  that  were  reached  about 
the  effect  of  PFM  on  each  attribute  were  consistent  between 
graders.  It  is  therefore  concluded  that  while  the  variable 
grader  (GDR)  had  a  statistically  significant  effect  on  the 
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scores  awarded,  it  did  not  have  an  effect  on  the  conclusions 
that  would  be  reached. 


Table  13 .  Mean  Scores  Awarded  by  Grader 


ATTRIBUTES 

GRADER  J 
CTL  PFM 

GRADER  S 
CTL  PFM 

SOLUTION 

IMPLICATION 

75 

66 

67 

75 

STRUCTURAL 

CONSIDERATIONS 

37 

55 

n 

66 

STAKEHOLDERS 

15 

Bi 

m 

72 

LEVEL  OF  FOCUS 

61 

62 

53 

69 

PERFORMANCE 

IDENTIFICATION 

30 

56 

34 

60 

PERFORMANCE 

MEASURABILITY 

41 

64 

64 

89 

Neither  grader  awarded  scores  that  resulted  in  a  sig¬ 
nificant  difference  for  attribute  XI  (Solution  Implication) . 
However,  Grader  J  rated  PFM  slightly  below  CTL,  whereas 
Grader  S  rated  PFM  slightly  higher  than  CTL.  Even  though  no 
assignable  cause  can  be  determined,  the  reversed  rankings 
illustrate  that  personal  values  can  affect  evaluation 


results . 
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solution  Structural  Stakeholders  Level  of  Performance  Performance 
implication  Consideration  Focus  identification  weasuraoi 1 ;t» 


ATTRIBUTES 

Figure  8.  Mean  Values  by  Grader  for  Six  Attributes 

4.5.5  Correlation  Between  Attributes 

The  correlation  between  scores  for  the  six  attributes  for 
total  sample  population  is  shown  in  Table  14.  A  strong 
positive  correlation  exists  between  four  of  the  attributes: 
Structural  Consideration,  Stakeholders,  Performance  Ident¬ 
ification  and  Performance  Measurability.  Tables  15  and  16 
display  the  correlation  between  the  six  attributes  for  the 
control  group  and  PFM  group  respectively.  In  the  control 
group,  very  low  correlation  exists  between  the  four  at¬ 
tributes.  In  the  PFM  group,  the  strong  correlation  remains. 


Table  14.  Correlation  Coefficients  Between  Attributes 


ATTRIBUTES 

SOLUTION 

IMPLICATION 

XI 

STRUCTURAL 

CONSIDERATIONS 

X2 

STAKEHOLDERS 

X3 

LEVEL  OF  FOCUS 

X4 

PERFORMANCE 

IDENTIFICATION 

X5 

PERFORMANCE 

MEASURABILITY 

X6 

XI  X2  X3  X4  X5  X6 


1.00  0.19  -0.02  *0.69  -0.04  -0.05 

1.00  *0.54  0.28  *0.83  *0.46 

1.00  0.11  *0.58  *0.64 

1.00  0.15 

1.00  1*0.55 


*Signif leant  at  a=0.10 

Table  15.  Correlation  Coefficients  for  Control  Group 


ATTRIBUTES 

SOLUTION 

IMPLICATION 

XI 

STRUCTURAL 

CONSIDERATIONS 

X2 

STAKEHOLDERS 

X3 

LEVEL  OF  FOCUS 

X4 

PERFORMANCE 

IDENTIFICATION 

X5 

PERFORMANCE 

MEASURABILITY 

X6 

XI 

X2 

X3 

X4 

X5 

1.00 

-0.20 

-0.19 

*0.70 

-0.33 

1.00 

0.07 

0.20 

0.40 

1.00 

-0.03 

*0.50 

1.00 

0.07 

1.00 

♦Significant  at  a=0.10 
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Table  16.  Correlation  Coefficients  for  PFM  Group 


ATTRIBUTES 

XI 

X2 

X3 

X4 

X5 

X6 

SOLUTION 

IMPLICATION 

XI 

1.00 

0.51 

0.14 

*0.74 

0.17 

0.20 

STRUCTURAL 

CONSIDERATIONS 

X2 

1.00 

0.45 

0.32 

*0.86 

*0.54 

STAKEHOLDERS 

X3 

1.00 

0.06 

0.34 

*0.76 

LEVEL  OF  FOCUS 

X4 

1.00 

0.06 

0.11 

PERFORMANCE 

IDENTIFICATION 

X5 

1.00 

*0.53 

PERFORMANCE 

MEASURABILITY 

X6 

1.00 

♦Significant  at  ar=0.10 


The  strong  correlation  in  the  PFM  group  coupled  with  the 
significant  improvement  in  the  same  four  categories  suggests 
that  when  the  problem  formulation  method  is  used,  the  four 
attributes  are  not  independent.  This  suggests  that  an 
improved  four-dimensional  model  could  be  defined  using 
Solution  Implication,  Level  of  Focus,  Performance  Identifica¬ 
tion  and  Performance  Measurability.  As  discussed  in  4.2,  the 
latter  two  attributes  would  address  the  same  concerns  as 
Stakeholders  and  Structural  Considerations,  but  from  a  more 
appropriate  perspective.  As  shown  in  Table  17,  aggregate 
scores  using  these  four  attributes  support  the  same  con¬ 


clusions  as  either  Sakman's  model  or  the  six-attribute  model. 


Table  17.  Improved  Four-Attribute  Model 

Source  Degrees  of  F  Value  Probability  of 
Freedom  a  Value  >  F 

PFM  1  24.94  <0.01  * 

GDR  1  13.84  <0.01  * 


♦Significant  at  a=0.10 
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5  CONCLUSIONS 


The  objective  of  this  research  was  to  improve  problem 
formulation  in  engineering  systems  design.  Several  reasons 
attest  to  the  importance  of  investigating  improvements  to 
problem  formulation.  Since  problem  formulation  occurs  in  the 
early  stages  of  the  design  process,  it  affects  all  succeeding 
phases  of  the  process.  Providing  well  defined  and  measurable 
boundaries  and  closure  to  a  problem  allows  the  designer  to 
focus  creative  efforts  within  those  areas  with  the  highest 
probability  of  solving  the  need  and  to  know  when  a  project 
is  complete.  Improvements  to  problem  formulation  reduce  the 
probability  of  solving  the  wrong  or  suboptimal  problem. 
Design,  including  problem  formulation,  is  a  decision-making 
process.  Generally,  the  amount  of  information  required  to 
change  a  decision  is  greater  than  the  amount  required  to  make 
an  original  decision.  Therefore,  improvements  in  problem 
formulation  should  improve  overall  design  efficiency. 

An  original  problem  formulation  method  was  developed  based 
on  needs  identified  during  an  investigation  of  when,  how  and 
why  information  and  knowledge  are  introduced  into  the 
engineering  systems  design  process.  The  method  recommends 
searching  along  five  directions  (time,  dimensions,  value, 
physical  matter,  and  people)  to  assist  in  identifying 
boundary  and  performance  criteria,  and  technical  performance 
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requirements  as  a  means  of  expressing  them.  The  method  was 

developed  by  combining  an  original  manner  of  expressing 

\ 

requirements  in  measurable  technical  terms  with  an  engineer¬ 
ing  adaptation  of  Dimensional  Analysis.  Dimensional  Analysis 
is  an  established  technique  for  general  problem  formulation. 
The  method  introduced  by  this  work  resulted  in  statistically 
significant  improvements  in  the  quality  of  problem  statements 
when  tested  in  a  controlled  task,  guided  response  experiment 
using  a  Multiple  Attribute  Value  model  for  evaluation. 

Performance  Identification  and  Performance  Measurability 
were  defined  as  two  new  essential  attributes  for  assessing 
improvements  in  problem  formulation.  It  is  recommended  that 
Performance  Identification  and  Performance  Measurability 
replace  Structural  Considerations  and  Stakeholders  as  two  at¬ 
tributes  of  problem  statement  quality  when  addressing 
engineering  systems  problems.  The  newly  defined  attributes 
more  precisely  identify  what  performance  is  required  to 
satisfy  the  identified  need  in  engineering  design  problems. 

The  improved  quality  ratings  were  obtained  without  pre¬ 
training  the  test  subjects.  Previous  work  using  the  same 
evaluation  method  incorporated  pre-training,  yet  failed  to 
detect  significant  differences  for  overall  problem  statement 
quality  between  the  general  problem  formulation  methods. 

The  improved  quality  ratings  were  not  sensitive  to  the 
weights  assigned  to  individual  attributes  when  combined  in 
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a  Multiple  Attribute  Value  model .  This  indicates  that  the 

improvements  are  sufficiently  substantial  to  be  tested  under 

\ 

various  models  of  problem  statement  quality. 

The  surveys  identified  reasons  that  information  outside 
one's  immediate  area  of  expertise  is  more  difficult  to 
obtain,  as  discussed  in  Appendix  B.  Identification  of 
perceived  reasons  is  an  important  first  step  toward  reducing 
the  difficulty. 

The  surveys  also  identified  industry  specific  categories 
and  sources  of  information  and  knowledge  that  are  important 
and  frequently  used  in  the  design  process.  The  identified 
sources  could  be  useful  in  developing  training  programs  for 
new  engineers,  or  in  determining  appropriate  mediums  for 
disseminating  information. 

The  sample  populations  of  the  surveys,  the  protocol  experi¬ 
ments,  and  the  problem  formulation  experiments  were  statisti¬ 
cally  adequate  in  size  but  limited  in  variety.  Similar 
surveys,  protocols  and  problem  formulation  experiments  in 
other  engineering  disciplines  and  industries  could  be 
conducted  to  further  generalize  the  findings.  On  the 
contrary,  the  general  method  could  be  tailored  to  specific 
disciplines  or  industries  by  expounding  on  the  xive  direc¬ 
tions  for  seeking  information. 

The  problem  formulation  experiment  was  not  preceded  by  any 
training  to  ensure  that  each  participant  had  some  baseline 
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understanding  of  what  constitutes  a  "good"  problem  statement. 
The  diverse  backgrounds  of  the  test  subjects,  including 
education  and  experience,  might  justify  pretraining  to 
establish  a  common  understanding  of  how  to  formulate  problems 
and  write  problem  statements.  The  variance  within  each 
treatment  group  was  consistent,  indicating  that  "poor"  and 
"good"  performers  were  randomly  distributed  between  the 
control  group  and  the  problem  formulation  method  group, 
^retraining  could  reduce  the  variances  within  treatments,  but 
its  effect  on  treatment  means  cannot  be  speculated  without 
further  study. 

The  demonstrated  improvement  in  problem  statement  quality 
does  not  ensure  improvement  in  the  ultimate  solution  to  the 
problem  or  in  design  efficiency.  Extending  the  evaluation 
procedure  to  include  the  remaining  steps  of  the  design 
process  would  be  required  to  evaluate  any  correlation  between 
the  quality  of  problem  formulation  and  either  the  quality  of 
solutions  or  the  efficiency  of  the  design  process. 

The  method  was  tested  under  controlled  conditions.  An 
extension  of  the  research  program  should  evaluate  its  effect 
under  corporate  conditions  using  either  case  studies  or 
concurrent  observation.  This  extension  could  address  the 
effect  of  problem  formulation  on  overall  design  efficiency 
by  evaluating  projects  from  inception  to  completion. 
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The  findings  suggest  two  additional  topics  for  further 
research.  First,  the  lack  of  significant  difference  in 
quality  of  the  problem  statements  between  the  practicing 
engineers  and  the  graduate  students  indicated  ir  this  study 
warrants  further  investigation.  Second,  the  surveys  of 
practicing  engineers  indicated  that  marketing  information  was 
both  the  most  difficult  to  obtain  and  one  of  most  critical 
categories.  Interdisciplinary  work  could  lead  to  the 
development  of  an  approach  that  ensures  that  marketing 
requirements  are  properly  reflected  during  engineering 
problem  formulation. 
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APPENDIX  A 

\ 

A. 1  Initial  Survey  Objectives 

A.  Identify  categories  of  information  topics  used  by  packag¬ 
ing  systems  designers.  [Behavior] 

A. 1 .  Determine  the  areas  in  which  information  is  required. 

A.  2.  Determine  which  information  is  most  difficult  to  obtain. 

B.  Identify  categories  of  information  sources  used  by  packag¬ 
ing  systems  designers.  [Behavior] 

B.l.  Determine  which  sources  are  used. 

B. 2.  Determine  which  sources  are  used  most  often. 

C.  Identify  categories  of  knowledge  topics  used  by  packaging 
systems  designers.  [Behavior] 

C. l.  Determine  the  types  of  knowledge  most  often  used. 

D.  Identify  categories  of  knowledge  sources  used  by  packaging 
systems  designers.  [Behavior] 

D. l.  Determine  where  knowledge  is  most  often  acquired. 

E.  Collect  sufficient  attributes  about  the  respondents  to 
test  hypotheses  about  the  effect  of  the  following  factors  on 
their  responses:  [Attributes] 

a.  experience 

b.  age 

c.  education  and  training 

d.  company  size  and  organization 


e.  company  purpose 
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F.  Determine  what  practicing  engineers  consider  to  be 
important  to  success:  [Attitudes,  Beliefs] 

F.l.  What  do  engineers  believe  are  the  attributes  required 
to  do  their  job  successfully? 

F. 2.  What  aspects  of  the  development  process  do  engineers 
believe  would  lead  to  improvements? 

G.  Determine  how  computers  are  used  in  the  development 
process.  [Behavior] 

NOTE:  Types  of  information  collected  using  surveys: 
Attitudes  -  What  people  say  they  want 

Beliefs  -  What  people  think  is  true 

Behavior  -  What  people  do 

Attributes  -  What  people  are 
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A. 2  Initial  Survey  and  Response  Data 

Note:  Numbers  indicate  totals  from  all  respondents 


Control  Number 


SECTION  I  -  GENERAL 


The  purpose  of  this  section  is  to  determine  the  range  of  professionals  who 
are  participating  in  the  research.  The  data  will  be  used  to  ensure  that  the 
research  includes  a  broad  range  of  participants,  so  that  the  results  will  not 
be  slanted  toward  any  segment  of  the  industry. 

1.  How  long  have  you  worked  in  the  packaging  or  packaging  related  positions? 

1  Less  than  two  years  _7  Ten  to  twenty  years 

10  Two  to  five  years  _6  Over  twenty  years 

2J  Five  tc  ten  years 


2.  What  is  your  current  position? 

36  Engineer  _4  Management 

J  Designer  _0  Marketing 

_0  Technical  Sales  Representative 
_4  Other  (Please  describe:  consultant,  owner  ) 


3.  What  is  the  highest  level  of  education  you  have  achieved  at  this  point  in  your 
career? 

__0  High  school  graduate  LI  Some  graduate  study 

_1  Some  undergraduate  college  _4  Masters  degree 

_2  Two  year  college  program  _0  Doctoral  Degree 

25  ..achelor  degree 

jj  Other  (Please  describe: _ ) 

4.  What  age  bracket  do  you  belong  to: 

_7  25  years  and  under  __5  46  to  55  years 

24  26  to  35  years  _2  56  years  and  over 

_7  36  to  45  years 

5.  During  your  career,  how  much  formal  training  directly  related  to  packaging 
systems  development  would  you  estimate  that  you  have  received0 

_7  less  than  1  month  J  7  to  12  months 

LI  1  to  6  months  L§  over  12  months 


6.  How  large  is  the  firm  in  which  you  are  currently  employed0 

_3  1  to  50  employees 
_4  50  to  500  employees 
38  Over  500  employees 

7.  If  your  firm  employs  over  500  people,  where  is  the  group  responsible  for 
packaging  systems  development  located0 

16  At  corporate  level 
19  At  division  level 

_5  Other  (Please  describe:  below  division  ) 


8.  Approximately  how  many  people  in  your  firm  are  directly  responsible  for 
packaging  system  development?  7 

9.  Which  one  of  the  following  classifications  best  describes  the  primary  way  in 
which  your  company  is  involved  in  packaging  systems: 

_3  Supplier 
J7  User 

_5  Other  (Please  describe:  consultant,  both  supplier  and  user) 


**************** *************** ************************************************ 

If  your  answer  to  question  9  identifies  your  company  as  a  Supplier,  please  go 
to  question  nuaber  15.  If  your  answer  to  question  9  identifies  your  company  as 
a  User  or  Other,  please  continue  with  question  10. 

******************************************************************************* 

10.  Which  of  the  following  classifications  describe  the  principle  products  of 
your  company?  You  aay  indicate  more  than  one. 

5  Food  and  beverage  1_1  Electrical  and  electronic 

1  Pharmaceutical  _4  Automotive 

6  Health  and  personal  care  _3  Paper  and  allied  products 

2  Chemicals  and  allied  products 

19  Other  (Please  describe: furni ture( 4 ) .  appl iancesl 4 ) ,  consulting! 3 ) , 

medical  equipment  (3).  general  merchandised )  .miscellaneous  (3) ) 

11.  Which  of  the  following  categories  best  describe  your  primary  customers?  You 
may  indicate  more  than  one. 

25  Consumer 

20  Industrial 
9  Military 

7  Other  (Please  describe:  medical,  consulting  > 

12.  Which  of  the  following  statements  best  describes  your  primary  customers? 
You  may  indicate  more  than  one. 

37  Established  or  long  term  customers 
19  New  or  short  term  customers 

24  Large  order  quantities 
28  Small  order  quantities 

28  Continuous  customer  demand  for  the  products 
16  Highly  fluctuating  customer  demand 

32  Short  order/ship  lead  time 
12  Long  order/ship  lead  time 

13.  Packaging  design  within  your  company  is  under  the  direct  supervision  of 
which  of  the  following: 

_6  Product  development  (including  8  4  D) 

_8  Manufacturing 
_3  Marketing 
16  Engineering 

_9  Other  (Please  describe:  quality,  packaging  ) 


14.  Including  the  supervisor  identified  in  question  13,  how  many  levels  of 
management  are  there  between  the  leader  of  the  packaging  group  and  the  company 
president? _ 4 _ 
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If  you  answered  questions  10  through  14,  please  go  to  question  20.  If  your 
company  is  a  Supplier,  please  continue  with  question  15. 

15.  Which  of  the  following  best  describes  your  company’s  products.  You  may 
indicate  more  than  one. 

_2  Packaging  aaterials  _2  Shipping  containers 

_1  Primary  containers  _1  Packaging  machinery 

_0  Packaging  components 

_0  Other  (Please  describe: _ ) 

16.  Which  of  the  following  best  describes  the  primary  materials 
used  in  your  company  product’s?  You  may  indicate  more  than  one. 

_2  Plastics  _0  Glass 

_0  Aluminum  _1  Steel 

_2  Paper,  paperboard  or  corrugated  board 

_0  Other  (Please  describe: _ ) 

17.  Which  of  the  following  classifications  describe  the  principle  products  of 
the  customers  you  supply?  You  may  indicate  more  than  one. 

Food  and  beverage  _2  Electrical  and  electronic 

_1  Pharmaceutical  __2  Automotive 

_1  Health  and  personal  care  _2  Paper  and  allied  products 

_2  Chemicals  and  allied  products 
_1  Other  (Please  describe:  medical  supplies  ) 

18.  Which  of  the  following  categories  best  describe  how  your  customers  use  the 
packages  you  supply?  You  may  indicate  more  than  one. 

_1  Consumer 
_4  Industrial 
_1  Military 

_0  Other  (Please  describe: _ ) 

19.  Which  of  the  following  statements  best  describes  your  primary  customers? 
You  may  indicate  more  than  one. 

_4  Established  or  long  term  customers 
_0  New  or  short  term  customers 

_3  Large  order  quantities 
_1  Small  order  quantities 

_4  Continuous  customer  demand  for  the  products 
_0  Highly  fluctuating  customer  demand 

_3  Short  order/ship  lead  time 
_2  Long  order/ship  lead  time 
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Note:  Numbers  in  questions  21,  24  and  27  indicate  unweighted 
totals  of  all  respondents. 

SECTION  II  -  INFORMATION  AND  KNOWLEDGE 

The  purpose  of  this  section  is  to  determine  the  range  of  categories  of 
information  and  knowledge  used  in  the  packaging  systems  development.  To  reduce 
the  amount  of  writing  required  for  each  question,  several  likely  responses  will 
be  provided,  followed  by  several  blank  lines.  Please  indicate  each  of  the  pre¬ 
printed  responses  you  wish  to  include  in  your  answer,  then  use  the  blank  lines 
to  print  additional  responses  which  apply  to  you  and  your  company. 

20.  When  you  begin  to  develop  a  new  packaging  system,  what  are  the  areas  in 
which  you  need  information?  Information  may  include  data  or  assumptions  from 
any  source. 

1  need  information  about: 

42  Product  to  be  packaged 
30  Product  manufacturing 
process  or  equipment 
32  Package  manufacturing 
process  or  equipment 
*2  Contractual 
*2  Logistics 

*6  Production  rate/auanti ty 
(*  indicates  write-in  response) 

21.  Which  of  the  information  listed  in  question  20  do  you  have  the  most 
difficulty  obtaining?  Rank  up  to  five  topics,  beginning  with  the  most  difficult. 

11)  Marketing  (29) _  14)  Regulations  and  laws  (17) 

(2)  Product  (26) _  (5)  Product  manuf.  process  (14) 

( 3 )  Distribution  (22) _  ( 6 )  Package  manuf.  process  (13) 

1 ' )  Package  materials  (13) 

22.  What  is  the  most  unusual  piece  of  information  you  ever  had  to  find  in  order 

to  develop  a  workable  packaging  system?  Anecdotal  information _ 


33  Distribution  system 

34  Marketing  requirements 
15  Competition 

38  Packaging  materials 
29  Regulations  and  laws 


How  did  you  obtain  that  unusual  information? 


23.  Which  sources  do  you  routinely  use  to  gather  the  information  you  need  to 
develop  a  packaging  system? 

26  Trade  journals  J>  Consultants 

21  Reference  books  26  Experts  inside  of  your  firm 

31  Vendor  specifications  27  Experts  outside  of  your  firm 

_8  Proprietary  sources  14  Seminars 

_7  Other  (Please  describe:  regulatory  sources,  military  specifications  ) 


24.  From  question  23,  rank  the  three  information  sources  you  use  most  oiten: 

1 .  Vendor  specifications  (30)  4 .  Expert  outside  firm  (16) 

2 .  Expert  inside  firm  (20)  5 .  Reference  books  ( 14  ) _ 

3.  Trade  journals  ( 19) _ 


25.  What  is  the  most  unusual  information  source  you  ever  used? 

Anecdotal  information _ 
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-0.  knowledge  may  be  defined  as  the  tools  or  methods  which  you  use  1  n 
conjunction  w 1 th  the  information  you  have  fathered  in  order  to  make  decisions 
or  develop  your  packaging  system.  Please  list  the  types  of'  knowledge  you  most 
often  use  in  developing  packaging  systems. 


40  Common  sense 
16  Rules  of  thumb 
2j  Design  guides 
14  Engineering  equations 
24  Standard  procedures 
32  Industry  practice 
* 2  Consumer  feedback 

*2  Cost  analysis _ 

*1  Machine  capability  study 
• 

27.  From  question  26,  which  forms  of  k 
up  to  three,  beginning  with  the  most  o 


4  Graph 

33  Laboratory  testing 
37  Comparison  to  previous 
packaging  problems 
13  Computer  simulation 
*  1  Experience _ 


*  write-in  response 

nowiedge  do  you  use  the  most  often?  List 
ften  used. 


1 .  Laboratory  testing  (28) _ 

2.  Common  sense  (27) _ 

3.  Comparison  to  previous  packaging  problems  (21) 

4.  Industry  practice  (18) 

5 •  Design  guides  (10) 

28.  Select  the  one  response  which  best  describes  where  or  how  you  acquired  the 
knowledge  that  helps  you  do  your  job. 


_3  Seminars 
_1  Short  courses 

16  Formal  education  (including  college  or  university! 

28  Experience  on  the  job 

_4  From  a  more  experienced  packaging  professional 
_7  Learning  on  your  own 

_0  Other  (Please  describe: _  ) 

SECTION  111  -  THE  PROFESSION 


29.  Assume  that  you  are  responsible  for  hiring  a  recent  college  graduate  that 
will  be  trained  to  work  as  an  engineer  with  responsibility  for  packaging  systems 
development.  What  essential  attributes  would  you  look  for  in  the  candidates? 


Technical  education  or  skill _ ( 20 ) 

Communication  skills _ ( 15  ) 

Ability  to  work  with  others _ ( 13) 

Cooperative  or  work  experience _ ( 10 ) 

Common  sense _ ( 10 ) 

Enthusiastic _ (  9 ) 

Creativity _ (  8 ) 


30.  Assume  you  have  just  hired  a  new  engineer  to  be  trained  for  the  position 
described  in  question  31.  What  professional  advice  would  you  offer  during  the 
first  week  of  employment? 


Ask  questions _ ( 13) 

Meet  people  in  the  organization _ (  8) 

Listen _ LJU 

Get  involved  quickly _ (  5  ) 

Stay  technically  current _ (  5 ) 


31.  Assume  that  you  have  just  been  given  responsibility  lor  a  new  packaging 
system  development  project.  What  are  the  first  three  things  you  will  do 

1  •  Obtain  information  aboput  product,  aarket  and  distribution  ___  _ _ 


2. 

Establish  schedule  and  budget 

3. 

Organize  a  design  team 

32. 

What  would  make  the  process  of  packaging  system  development 

easier  for  you? 

Clear  and  non-changing  reauirenents 

(9) 

Computer  aids 

(9) 

Getting  involved  earlier  in  the  design  process 

16) 

More  time 

(5) 

Product  samples  available  earlier  in  developaent  process 

( 5 ) 

Clear  Data 

(4) 

Formalized  procedures 

(3) 

Feedback  from  the  field 

(3) 

33.  How  often  do  you  use  a  computer  to  assist  you  in  developing  packaging 
systems? 

_7  I  use  a  computer  constantly 
22  l  use  a  computer  at  least  daily 
_8  1  use  a  computer  at  least  once  a  week 
5  I  rarely  use  a  computer 
J  I  do  not  use  a  computer 

34.  If  you  do  use  a  computer,  what  are  the  most  common  applications  tnat  you 
use  in  packaging  system  development?  You  nay  indicate  more  than  one. 

26  Word  processing 

26  Specifications 

15  Access  to  data  banks 

27  Drafting  or  drawing 

17  Simulations  or  calculations 

_8  Other  (Please  describe: Budgeting  or  scheduling) 


SUMMARY 


Thank  you  for  participating  in  this  initial  phase  of  our  research.  A 
summary  of  the  results  will  be  provided  to  you  in  appreciation  for  your 
assistance,  lou  should  expect  to  receive  the  summary  no  later  than  the  end  of 
September  1989. 

Based  on  the  results  of  this  initial  phase  of  research,  we  will  be  seeking 
assistance  in  gathering  more  specific  data  about  the  information  and  knowledge 
used  in  the  development  of  packaging  systems.  It  is  anticipated  that  this  next 
phase  will  begin  no  later  than  September  1989  and  will  require  approximately  the 
same  amount  of  time  investment  from  the  participants.  Again,  we  will  provide 
a  summary  of  the  results  of  the  next  phase  to  all  interested  participants. 

The  next  phase  may  be  distributed  both  in  printed  form  and  on  a  5  1/4  inch 
diskette  compatible  for  use  on  any  personal  computer  using  MS-DOS  operating 
system.  This  should  reduce  the  time  required  for  participants  as  well  as 
assisting  in  data  analysis. 

Please  indicate  below  if  you  are  interested  in  receiving  the  research 
summary  or  if  you  are  willing  to  participate  in  the  next  phase  of  the  research. 


_  Please  send  a  copy  of  the  research  summary  to  me  at  the  address  shown  below. 

_  Yes,  I  am  interested  in  participating  in  the  next  phase  of  research.  I 

would  prefer  to  complete  the  next  phase  of  research  using: 

_  5  1/4"  diskette 

_  Printed  format 

s***********************s*********************s*************s******************** 

Please  attach  your  business  card  or  print  the  address  information  requested 

below. 

Name: _ 

Company _ 

Address _ 

City _ State _ Zip  Code _ 

Telephone  Number_| _ ] _ FAX  Number _ 
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APPENDIX  B 

B.l  Objectives  of  the  Follow-up  Survey 

Purpose; 

1.  Determine  if  engineers  currently  apply  a  systems  approach 
to  engineering  design. 

2.  Determine  what  effect  a  systems  approach  has  on  the 
identification  and  acquisition  of  information  and  knowledge 
during  the  design  process. 

Objectives:  Within  the  survey  population,  determine  the 

current  state  of  three  study  topics  by  seeking  answers  to  the 
questions  listed  below.  Numbers  in  parentheses  refer  to  the 
survey  questions  which  provide  indicators  for  answering  each 
topic. 

TOPIC  A:  The  use  of  a  systems  approach  to  engineering 

design. 

A.l.  Do  engineers  believe  they  use  a  systems  approach  to 
engineering  design?  (Questions  1,2) 

A. 2.  How  well  do  engineers  actually  use  a  systems  approach 
to  engineering  design?  (Questions  4,5,6,7,8,9,12,13,14,16,21) 
A. 3.  Does  the  information  and  knowledge  sought  by  engineers 
indicate  the  use  of  a  systems  approach  to  engineering  design? 
(Questions  6,7,8,9,12,13) 
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TOPIC  B:  The  use  of  clearly  stated  technical  performance 
requirements  in  the  design  process. 

v 

B.l.  What  do  engineers  consider  to  be  requirements? 
(Questions  5,6,7,12,16,18,19) 

B.2.  Do  engineers  write  technical  performance  requirements 
from  marketing  requirements?  (Questions  3,5,8,9,10,13) 

B.3.  Who  participates  in  writing  and  approving  technical 
performance  requirements?  (Questions  14,15) 

B.4.  At  what  point  in  the  design  process  are  technical 
performance  requirements  written?  (Questions  3,10) 

B.5.  What  elements  are  included  in  technical  performance 
requirement  statements?  (Question  16) 

B.6.  What  internal  and  external  marketing  requirements  are 
considered  in  writing  technical  performance  requirements? 
(Questions  18,19) 

B.7.  How  do  engineers  insure  that  requirements  are  satisfied 
during  the  design  process?  (Question  21) 

B.8.  What  happens  in  the  design  process  if  all  requirements 
cannot  be  satisfied?  (Question  17) 

B.9.  After  design  is  complete,  do  engineers  use  performance 
specifications  or  material  property  specifications  to  docu¬ 
ment  manufacturing  process  and  supplier  requirements? 
(Question  20) 
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TOPIC  C:  The  relative  difficulty  of  obtaining  information 

inside  and  outside  the  engineer's  immediate  area  of  expertise 

\ 

during  the  design  process. 

C.l.  Is  information  outside  of  the  immediate  area  of 
expertise  more  difficult  to  identify  and  obtain?  (Question 
11,22) 

C.2.  Is  an  increase  in  difficulty  due  to: 

(Questions  4,5,6,7,8,9,1,22) 

a.  lack  of  subject  knowledge,  either  background  under¬ 
standing  or  knowledge  specific  to  the  design  project? 

b.  time  and  cost  constraints  of  the  design  project? 

c.  lack  of  common  language? 

C.3.  With  whom  do  engineers  most  often  communicate  to  obtain 
knowledge  and  information?  (Question  9,11,13,14,22) 

C.4.  What  information  and  knowledge  is  most  important 
(Questions  4, 5, 6, 7, 8, 9, ,12,16,18,19) 


92 


B.2  Follow-up  Survey  and  Response  Data 

Control  Number _ 

Packaging  Systems  Development 
Research  Survey 

1.  What  do  you  design?  Mark  all  applicable  responses. 

Sam 

29  Packages  (primary,  secondary,  tertiary,  etc.) 

l  s  Packaging  operations 

u  Distribution  (shipping,  warehousing,  handling) 

i ;  Packaging  systems 

6  Packaging  manufacturing  equipment 

lo  Package  manufacturing  process 

!  Other  (Please  specify: _ ) 


2.  String  together  terms  from  the  following  alphabetical  list  to 
describe  your  job  responsibilities,  (e.g.  <design>  <packaging> 
<containers>) 


Sum 

Sum 

Sum 

7 

a. 

Consult 

i.  Machinery 

0 

q- 

Scientist 

3 

b. 

Container 

4 

j .  Manage 

3 

r. 

Supervise 

25 

c. 

Design 

3 

k.  Manufacture ( ing) 

10 

s. 

System 

14 

d. 

Develop 

6 

1.  Materials 

-> 

t. 

Technical 

5 

e. 

Direct 

27 

m.  Package (ing) 

1 

u. 

Technology 

2 

f. 

Distribution 

3 

n.  Prototype 

15 

v. 

Test (ing) 

14 

g- 

Engineer 

4 

o.  Quality 

w. 

other 

3 

h. 

Equipment 

1 

p.  Science 

>  < 

> 

<  >  < 

>  < 

> 

3.  In  what  relative  sequence  do  the  following  occur  in  solving  a 
packaging  problem?  Use  "1"  to  indicate  the  first  event,  "2”  the 
second,  etc.  Include  all  applicable  items. 

Sum  Ave 

98  ~i,4  Generate  feasible  solutions 

91  3.:  Translate  requirements  into  technical  terms 

34  i  .2  Determine  customer  needs 

62  2 .  i  Determine  packaging  requirements 


4.  How  important  are  the  following  in  the  design  process?  Rate 
each  topic  that  is  applicable  to  your  work. 


0 _ 1 _ l 

Not 

Important 

Sum  Ave 


2 _ 1 

Very 

Important 


1  18 
105 
100 
55 
52 
85 
96 


3.9  Defining  the  problem 

3.5  Analyzing  possible  solution 
3.3  Generating  possible  solutions 
2,o  Making  assumptions 

i  ,q  Management  review  of  the  design  in  progress 

2.9  Obtaining  input  from  other  departments 

2 . 2  Determining  how  your  design  problem  affects  and  is 

affected  bv  outside  sources 
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5.  Which  of  the  following  describe  the  design  requirements  from 
which  you  normally  work  in  developing  solutions  to  packaging 
problems?  Mark  all  applicable  responses. 

Sam 


21 

21 

n  •> 


11 


Derived  from  marketing  requirements 

Based  on  previous  in-house  packaging  solutions 

Include  manufacturing  process  or  equipment  requirements 

Written  in  technical  terms 

Specify  permissible  materials  to  be  considered 
Specify  performance  requirements,  but  not  how  to 
achieve  them 

Include  ranges  or  tolerances  for  specified  values 
Based  on  in-house  capabilities  and  limitations 
Focused  on  what  the  customer  needs 
Include  budget  or  cost  limitations 
Include  personnel  availability  or  limitations 


6.  From  responses  to  the  previous  survey,  listed  below  are  the 
areas  in  which  information  is  needed.  How  important  is  each  area 
to  your  design  work? 


o _ 1 _ 2 _ ' _ 3 _ 4 


Not 

Very 

Important 

Important 

Sum 

Ave 

117 

— 1-.3. 

Product  to  be  packaged 

69 

2.3 

Product  manufacturing  process  or 

equipmert 

83 

2.8 

Package  manufacturing  process  or 

equipment 

99 

3.2 

Distribution  system 

88 

-.1,0- 

Marketing  requirements 

53 

-1,9- 

What  your  competition  is  doing 

98 

3.2 

Packaging  materials 

80 

2.7 

Regulations  and  laws 

47 

1.9 

Contractual  requirements 

93 

3.1 

Rate  or  quantity  of  production 

75 

2.5 

Logistical  considerations 

84 

3.0 

Project  schedule 

85 

2.7 

Project  budget 

3 

■  3.0. 

Other(Please  specify: 

) 

7. 

Name 

or  describe  a  product  for  which  you  might  be  expected  to 

design  packaging: 


Write  the  first  three  questions  about  the  design  problem  that  you 
would  like  to  have  answered.  Please  respond  spontaneously. 


Sum  Topic 


04}  . 

Distribution  Svstem 

(2) 

(12) 

Package 

OD... 

Production  Ouantitv 

(3) 

(11)  _ 

End  Use  Environment 

—  W_ 

Product  Process 

(61  Production  Schedule 

2 


8.  How  important  are  the  following  in  the  design  process?  Rate 
each  applicable  topic. 


0 _ 

Not 

Important 


_ 4 

Very 

Important 


Sum  Ave 

74  2.6  Verifying  or  challenging  assumptions 

114  2,8  Determining  the  design  requirements 

73  7 . 4  Communicating  the  problem  and  solution  to  non-engineers 

loo  2,4  Gathering  information 

83  2.9  Determining  why  the  design  problem  must  be  solved  (what 

is  the  purpose) 

73  2.4  Obtaining  approval  for  the  design  requirements 

72  2.4  Translating  qualitative  requirements  into  quantitative 

technical  terms 


9.  Which  of  the  following  are  part  of  your  design  process' 
all  applicable  responses. 

Discuss  the  packaging  requirements  with: 

Sum  19  Customer 
?4  Marketing 
1 2  Sales 

?7  Product  scientist  or  product  engineer 
17  other  engineers 
29  Manufacturing 
a  Financial  planners 
12  Regulatory  advisors 

_ «_  Others  (Please  specify: _ 


Document  the  packaging  requirements: 

Sum  _ 3_  formally,  before  you  initiate  solutions 

is  as  a  routine  part  of  developing  solutions 

_ formally  as  you  develop  solutions 

18  once  the  solutions  are  developed 


Attempt  to  quantify  the  packaging  requirements 
Sun  14  before  you  develop  solutions 
is  as  you  develop  solutions 


10.  In  what  relative  sequence  do  the  following  occur  in  solving 
a  packaging  problem?  Use  "1"  to  indicate  the  first  event,  "2"  the 
second,  etc.  Include  only  those  items  which  are  applicable. 

Sum 

62  Determine  packaging  requirements 
35  Collect  information  about  the  product 

so  Collect  information  about  available  packaging  equipment 
and  processes 

97  Translate  requirements  into  technical  terms 
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11.  From  the  previous  survey,  the  following  seven  areas  were 
identified  as  the  most  difficult  for  obtaining  information.  Please 
indicate  all  the  reasons  you  have  difficulty  obtaining  information 
about: 


Marketing  of  the  product 


Sum 

] o  a.  Lack  of  time 


_ h_  b. 

_ c* 

12  d. 
Q  • 

-Li.  f* 
g. 


Too  costly 

Knowing  whom  to  ask 

Asking  the  right  questions 

Understanding  the  answers 

Information  not  available 

Do  not  use  the  same  terminology 

Someone  is  uncooperative 

Other  (Please  explain  International  Barriers _ ) 

Does  not  apply,  information  is  easily  obtained 


Product  to 

Sum 


10  a. 


_UL  j- 


Lack  of  time 

Too  costly 

Knowing  whom  to  ask 

Asking  the  right  questions 

Understanding  the  answers 

Information  not  available 

Do  not  use  the  same  terminology 

Someone  is  uncooperative 

other  (Please  explain  Prototype  Not  Available _ ) 

Does  not  apply,  information  is  easily  obtained 


_ l_  b.  Too  costly 

; ;  c.  Knowing  whom  to  ask 

_ 2_  Asking  the  right  questions 

_ e.  Understanding  the  answers 

ii  f.  Information  not  available 


_JJL  j* 


Do  not  use  the  same  terminology 
Someone  is  uncooperative 

Other  (Please  explain  People  Do  Not  Know  Answer _ ) 

Does  not  apply,  information  is  easily  obtained 


a. 

b. 


c. 

d. 


_LL  j* 


Lack  of  time 

Too  costly 

Knowing  whom  to  ask 

Asking  the  right  questions 

Understanding  the  answers 

Information  not  available 

Do  not  use  the  same  terminology 

Someone  is  uncooperative 

Other  (Please  explain  _ _ _ ) 

Does  not  apply,  information  is  easily  obtained 
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Product  manufacturing  process 

Sum 


_JA.  j* 


Lack  of  tine 

Too  costly 

Knowing  whom  to  ask 

Asking  the  right  questions 

Understanding  the  answers 

Information  not  available 

Do  not  use  the  same  terminology 

Someone  is  uncooperative 

Other  (Please  explain_  Process  Has  ~»'uL  Seen  Determined. 

Does  not  apply,  information  is  easily  obtained 


) 


Packaging  materials 


a. 

b. 

c. 

d. 

e. 

f . 

g- 

h. 

i. 

j  • 


Lack  of  time 

Too  costly 

Knowing  whom  to  ask 

Asking  the  right  questions 

Understanding  the  answers 

Information  not  available 

Do  not  use  the  same  terminology 

Someone  is  uncooperative 

Other  (Please  explain  Ansvers  No  Known,  Interference 

Does  not  apply,  information  is  easily  obtained 


) 


Packaging  operations 


Sum 


a. 

b. 

c. 

d. 

e. 

f . 

g* 

h. 

i . 

j- 


Lack  of  time 

Too  costly  . 

Knowing  whom  to  ask 
Asking  the  right  questions 
Understanding  the  answers 
Information  not  available 
Do  not  use  the  same  terminology 
Someone  is  uncooperative 

Other  (Please  explain  Poor  Communications _ 

Does  not  apply,  information  is  easily  obtained 


) 


12.  How  often  do  you  need  to  seek  information  relating  to  a 
packaging  system  development  project  from  the  following? 


0 

1 

2 

3  4 

Never 

Sum  Ave 

Sum 

Ave 

On  Every 
Pro j  ect 

69 

2.6 

Marketing 

61 

Distribution 

50 

2.1 

Customer 

70 

2-5 

Management 

96 

3,3 

Manufacturing  or 
production 

82 

-  -  9 

Manufacturing  and  process 
engineers 

62 

2.6 

Quality  Assurance 

79 

3-8 

Vendors 

29 

9 

Li 

3.0 

Regulatory  advisors  ^8 
Others  (Please  specify”: _ 

1  -4 

Financial  planners 

) 

5 
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13.  At  some  point  in  a  design  project,  quantitative  values  (e.g., 
allowable  permeance,  size  limitations,  probable  drop  height)  are 
assigned  to  various  packaging  requirements.  How  often  do  you 
discuss  the  quantitative  values  of  packaging  requirements  with  each 
of  the  following? 


Never 

Sum  Ave 


Sum  Ave 


On  Every 
Project 


49  1.9  Marketing  o5  2.4  Distribution 

36  1.8  Customer  62  ; . 3  Management 

63  2.3  Manufacturing  or  69  2 .4  Manufacturing  and  process 

production  engineers 

69  2 .  s  Quality  Assurance  71  2 .4  Vendors 

23  1 . 1  Regulatory  advisors  12  0,9  Financial  planners 

0  0  Others  (Please  describe _ ) 


14.  How  much  does  each  of  the  following  contribute  when  you  are 
developing  packaging  requirements? 


Does  not 
Contribute 


Contributes 
the  most 


Sum  Ave 

78  3.0  Design  engineering 

44  '  .a  Customer 

6 1  2.3  Marketing 

23  i .?  Regulatory  advisors 

70  2.4  Process  or  equipment  engineering 

54  7.0  Quality  assurance 

43  '  -3  Management 

16  Q-9  Financial  planners 

82  •> .«  Manufacturing  or  production 

2  7  .n  Others  (Please  specify;  Purrh.nsir.g _ ) 


15.  Are  packaging  requirements  formalized  and  signed  by  approving 
authorities?  (YES)  (NO) 

Sum  22  9 

If  yes,  which  of  the  following  signatures  would  normally  appear  on 
the  document?  Select  all  applicable  responses. 

Sum 

12  Design  engineering 
2  Customer 


_ l_  Marketing 

_ Regulatory  advisors 

_ Process  or  equipment  engineering 

1 1  Quality  assurance 

12  Management 

_ l_  Financial  planners 

_ 1_  Manufacturing  or  production 

_ Others  (Please  specify;  Packaging  Engineer.  Material  Manager 


e. 
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16.  Which  of  the  following  elements  are  normally  included  in  the 
packaging  design  requirements  from  which  you  generate  solutions? 
Mark  all  applicable  responses. 


Sum 


Available  packaging  materials 
Available  packaging  equipment 
Available  personnel 
Cost  constraints 
Time  constraints 
Existing  patents 
Reference  to  your  competition 
Level  of  protection  required 

List  separating  packaging  characteristics  into  required 
and  desired  categories 


Tolerances  of  performance 

Conditions  under  which  the  product  must  be  protected 
Total  quantity  required 
Production  rate 

Other(Please  describe:  weights  And  sizes _ ) 


17.  Which  one  of  the  following  best  describes  the  action  taken 
when  one  or  more  packaging  requirements  cannot  be  satisfied? 


Sum 


Terminate  the  development  project 

Use  your  engineering  judgement  to  amend  the  packaging 
requirement 

Notify  the  source  of  the  original  packaging 
requirements 

Formally  renegotiate  the  packaging  requirements 
with  their  original  source 


18.  Which  of  the  following  internal  factors  are  normally 
considered  when  packaging  requirements  or  constraints  are  devel¬ 
oped?  Mark  all  applicable  answers. 


Sum 


Packaging  equipment  currently  available  in  the  company 
Personnel  currently  employed 
Company  financial  posture 
Program  budget 

Previous  solutions  to  similar  problems 
Packaging  materials  previously  used 

Other  (Please  describe:  Materials,  Special  Promotions _ ) 


19.  Which  of  the  following  external  factors  are  normally 
considered  when  packaging  requirements  or  constraints  are  devel¬ 
oped?  Mark  all  applicable  answers. 

Sum 


Existing  technology 

What  the  competition  is  doing 

Existing  patents 

Vendor  recommended  materials 

Equipment  available 

Personnel  available 

Other  (Please  describe:  Infprn.-if-innai  Materials.  Leeislatlpn  ) 
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20.  Which  one  of  the  following  statements  best  describes  the 
specifications  you  provide  to  your  vendors? 

sum 

2  3  Specifies  the  required  physical  properties 
(i.e.,  material  and  design  specifications) 

_ l_  Specifies  the  required  performance  in  technical  tern'' 

(i.e.,  allowable  permeance,  compressive  strength) 

_ 2_  Specifies  the  required  performance  in  non-technical  terms 

(i.e.,  strong,  leakproof,  lightweight) 


21.  How  do  you  insure  that  your  packaging  designs  meet  the  needs 
of  your  customer? 

Sum  Method 

_ Customer  Feedback _ _ 


f  1 2 1  _ Test  in  c 


(81  Customer  Surveys 


(81 _ Audits 


22.  How  difficult  is  it  to  obtain  needed  information  for  designs 
from  the  following?  Rate  each  applicable  response. 


0 _ 1 _ 2 _ 2 _ 4 

Not  Very 

Difficult  Difficult 


Sum 

36 

Ave 

Marketing 

Sun 

22 

Ave 

1 .8 

Distribution 

52 

2 . 5 

Customer 

56 

i 

Management 

55 

-•  *> 

Manufacturing  or 

35 

*  .  *4 

Manufacturing  and  process 

27 

n  *■> 

production 

Quality  Assurance 

34 

i .  3 

engineers 

Vendors 

39 

i  .  9 

Regulatory  advisors 

28 

2  .  C 

Financial  planners 

- 

-  .0 

Others (Please  specify: 

Dt-iier. 

Eneir.eer  ) 

Thank  you  again  for  investing  your  time  and  sharing  your 
expertise.  I  will  mail  you  the  research  summary  soon. 

Charlie  Ennis 

Department  of  Mechanical  Engineering 
Texas  A&M  University 
College  Station,  TX  77843-3123 
(409)  845-1597 
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B.3  Analysis  of  the  Follow-up  Survey 

Analysis  of  individual  questions  is  based  on  one-way  or 
two-way  contingency  tables,  or  plots  of  frequency  distribu¬ 
tions  or  relative  frequency  distributions.  The  mode  of  the 
responses  is  the  primary  statistical  tool  used  in  evaluation. 
The  essence  of  each  question  is  provided  in  brackets. 

TOPIC  A:  The  use  of  a  systems  approach  to  engineering  design. 

A.l.  Do  engineers  believe  they  use  a  systems  approach  to 
engineering  design?  (Questions  1,2) 

Ql:  [What  do  you  design?] 

Approximately  48%  of  the  respondents  indicated  that  they 
design  packaging  systems,  packaging  operations  or  distribu¬ 
tion,  whereas  94%  indicated  that  they  design  packages. 

Q2 :  [What  are  your  job  responsibilities?] 

Less  than  one  third  of  the  respondents  used  the  term  "sys¬ 
tems"  in  describing  their  job  responsibilities. 

Analysis:  The  respondents  do  not  strongly  refer  to  themselves 
as  systems  designers.  However,  tne  results  are  not  sig¬ 
nificant  enough  to  state  that  the  respondents  do  not  consider 
themselves  systems  designers. 
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A.  2.  How  well  do  engineers  actually  use  a  systems  approach 
to  engineering  design?  (Questions  4,5,6,7,9,12,13,14,16) 

Q4 :  [How  important  are  the  following  in  the  design  process?] 
"Defining  the  problem"  was  rated  as  the  most  important  topic. 
"Determining  how  the  design  problem  affects  and  is  affected 
by  outside  sources"  was  rated  by  respondents  as  important  as 
"generating  solutions"  and  "analyzing  solutions."  This 
supports  a  systems  approach. 

Q5:  [How  do  you  describe  design  requirements?] 

Q6:  [How  important  is  information  in  various  areas?] 

Q12 :  [How  frequently  is  information  sought  in  various  areas?] 
Q14:  [Who  contributes  to  developing  design  requirements?] 

Relatively  uniform  distribution  among  topics  and  sources  of 
information  indicates  use  of  a  systems  approach. 

Q7:  [What  information  is  sought  first  in  a  design  problem?] 

Information  was  sought  about  the  distribution,  end  user 
environment  and  the  product  as  often  as  it  was  sought  about 
the  package,  indicating  a  systems  approach. 
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Q9 :  [With  whom  are  design  requirements  discussed?] 

Q13:  [With  whom  are  technical  values  of  requirements  dis- 

v 

cussed?] 

Discussing  requirements  with  a  wide  range  of  departments 
supports  a  systems  approach. 

Q16:  [What  elements  are  included  in  design  requirements?] 

Wide  range  of  information  included  in  design  requirements 
supports  systems  approach. 

Analysis:  Knowingly  or  unknowing  ”,  the  respondents  indicate 
that  a  systems  approach  is  used  to  solve  packaging  problems. 

A. 3.  Do  the  information  and  knowledge  sought  by  engineers 
indicate  the  use  of  a  systems  approach  to  engineering  design? 
(Questions  6,7,9,12,13) 

Analysis:  As  discussed  in  question  A. 2.,  a  systems  approach 
is  indicated  by  the  information  sought. 

Topic  A  Conclusion:  The  respondents  do  use  a  systems  approach 
to  solving  packaging  problems,  although  they  may  not  formally 
recognize  it  as  such. 
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TOPIC  B:  The  use  of  clearly  stated  technical  performance 
requirements  in  the  design  process. 

V 

B.l.  What  do  engineers  consider  to  be  requirements? 
(Questions  5,16,18,19) 

Q5:  [How  do  you  describe  design  requirements?] 

Q16:  [What  elements  are  included  in  design  requirements?] 

Distribution  of  the  responses  indicates  that  design  require¬ 
ments  include  a  broad  spectrum  of  information  but  are  not 
necessarily  written  in  technical  terms.  Responses  to  both 
questions  indicate  that  personnel  availability  is  not  a 
strong  element  in  determining  requirements. 

Q18 :  [What  internal  factors  influence  design  requirements?] 
Q19 :  [What  external  factors  influence  design  requirements?] 

All  areas  listed,  except  personnel,  company  financial  posture 
and  existing  patents,  were  used  to  develop  packaging  require¬ 
ments  . 

Analysis:  An  appropriately  broad  range  of  performance  and 
technical  elements  are  considered  in  developing  the  packaging 
requirements . 
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B.2.  Do  engineers  write  technical  performance  requirements 
from  marketing  requirements?  (Questions  5,8,13) 

V 

Q5:  [How  do  you  describe  design  requirements?] 

The  highest  number  of  responses  (81%)  indicated  that  the 
design  requirements  were  derived  from  marketing  requirements. 
Six  other  topics  had  relative  frequencies  above  60%,  indicat¬ 
ing  that  "marketing  requirements"  is  only  one  of  several 
important  considerations  in  developing  design  requirements. 

Q8:  [How  important  are  various  activities  in  the  design 
process?] 

"Translating  requirements  into  technical  terms"  was  one  of 
the  least  important  aspects  of  the  design  process,  sig¬ 
nificantly  below  "determining  design  requirements"  and 
"gathering  information." 

Q13:  [With  whom  are  technical  values  of  requirements  dis¬ 
cussed?] 

On  a  scale  of  0  to  4,  quantitative  values  for  design  require¬ 
ments  were  discussed  with  marketing  less  often  (average 
rating  1.9)  than  with  management  (2.3)  or  other  technical 
functions  (2.3  to  2.5). 
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Analysis:  There  is  no  strong  indication  that  the  respondents 
deliberately  translate  marketing  requirements  into  technical 
terms,  although  marketing  requirements  are  considered  in 
developing  technical  design  requirements. 

B.3.  Who  participates  in  writing  and  approving  technical 
performance  requirements?  (Questions  14,15) 

Q14:  [Who  contributes  to  developing  design  requirements?] 

On  a  scale  of  0  to  4 ,  the  level  of  contribution  to  developing 
design  requirements  was  highest  for  design  engineering  (3.0) , 
manufacturing  (2.8),  process  or  equipment  engineering  (2.4), 
marketing  (2.3),  quality  assurance  (2.0),  and  customer  (1.8). 
The  remaining  categories  ranged  from  0.9  to  1.5. 

Q15:  [Who  approves  formalized  design  requirements?] 

Design  Engineering  (55%)  ,  Management  (55%)  and  Quality 
Assurance  (50%)  were  the  highest  three  frequencies.  Other 
categories  ranged  from  5%  to  32%  relative  frequency. 

Analysis:  A  wide  range  of  participants  contribute  to  the 
development  of  design  requirements  (Q14) ,  but  only  design 
engineering,  quality  assurance  and  management  were  consis¬ 
tently  listed  as  signatures  on  a  formal  requirements  docu¬ 
ment.  It  appears  that  approval  of  technical  requirements  is 
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kept  primarily  within  the  domain  of  technical  people  and 
management . 

B.4.  At  what  time  in  the  design  process  are  technical 
performance  requirements  written?  (Questions  3,9,10) 

Q3  &  Q10:  [In  what  relative  sequence  do  selected  design 
activities  occur?] 

The  translation  of  requirements  into  technical  terms  was  done 
very  late  in  the  design  process.  From  question  3,  an  equal 
percentage  (45%)  of  respondents  indicated  that  the  transla¬ 
tion  occurred  just  before  or  just  after  the  generation  of 
feasible  solutions. 

Q9:  [With  whom  are  design  requirements  discussed?] 

Design  requirements  are  developed  either  during  or  after,  but 
seldom  before  the  generation  of  feasible  solutions.  Sixty 
percent  of  the  respondents  attempt  to  quantify  design 
requirements  as  they  develop  solutions,  compared  to  forty- 
four  percent  attempting  to  quantify  them  before.  Some 
respondents  indicated  both  as  appropriate  answers. 

Analysis:  There  is  no  evidence  to  indicate  that  technical 
performance  requirements  are  developed  prior  to  the  genera¬ 
tion  of  feasible  solutions.  There  is  no  evidence  to  indicate 
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that  the  translation  of  requirements  into  technical  terms  is 
a  deliberate  step  in  the  design  process. 

B.5.  What  elements  are  included  in  technical  performance 
requirement  statements?  (Question  16) 

Q16:  [What  elements  are  included  in  design  requirements?] 

Analysis:  Respondents  identified  a  relatively  uniform 

distribution  of  categories  related  to  packaging  and  produc¬ 
tion,  to  include  cost,  time  constraints  and  quantities. 
Desired  and  required  characteristics  are  not  listed  separate¬ 
ly,  and  tolerances  of  performance  are  not  a  strong  considera¬ 
tion. 


B.6.  What  internal  and  external  marketing  requirements  are 
considered  in  writing  technical  performance  requirements? 
(Questions  18,19) 

Q18:  [What  internal  factors  influence  design  requirements?] 

Equipment  (87%),  materials  (84%),  and  previous  solutions 
(84%)  were  considered  most  frequently,  followed  closely  by 
the  program  budget  (68%) .  Personnel  (42%)  and  company 
financial  posture  (29%)  were  considered  less  frequently. 


108 


Q19 :  [What  external  factors  influence  design  requirements?] 

Materials  (84%) ,  existing  technology  (84%)  and  equipment 

v 

available  (71%)  were  considered  most  frequently.  Competition 
(52%) ,  personnel  (39%) ,  and  existing  patents  (19%)  were 
considered  less  frequently. 

Analysis:  Emphasis  is  on  the  technical  issues  and  on  the 
specific  packaging  problem  at  hand.  More  indirect  topics  are 
not  considered. 

B.7.  How  do  engineers  insure  that  requirements  are  satisfied 
during  the  design  process?  (Question  21) 

Q21:  [How  do  you  insure  that  designs  meet  customer  needs?] 

Analysis:  Customer  feedback,  formal  and  informal,  and  testing 
of  the  packaging  and  materials  were  the  dominant  methods, 
accounting  for  over  50%  of  the  methods  used.  Testing  did  not 
indicate  any  assurance  that  the  package  met  the  needs,  only 
that  it  did  not  fail  to  meet  the  design  specifications. 
Customer  surveys  and  internal  company  audits  were  also  listed 
frequently.  Customer  satisfaction  with  varying  degrees  of 
formality  appears  to  be  the  primary  indicator  of  design 
adequacy. 
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B.8.  What  happens  in  the  design  process  if  all  requirements 
cannot  be  satisfied?  (Question  17) 

Q17:  [What  action  is  taken  when  a  design  requirement  cannot 
be  satisfied?] 

Analysis:  Formal  renegotiation  was  the  primary  method  (61%) , 
with  engineering  judgement  a  strong  second  (42%)  .  Both 
indicate  an  active  approach,  as  opposed  to  a  defensive 
posture  of  terminating  the  project  (0%)  or  only  notifying  the 
source  (26%) .  Response  indicates  the  desire  to  keep  the 
design  process  in  motion. 

B.9.  After  design  is  complete,  do  engineers  use  performance 
specifications  or  material  property  specifications  to  docu¬ 
ment  manufacturing  process  and  supplier  requirements? 
(Question  20) 

Q20:  [What  form  of  specifications  are  provided  to  vendors?] 

Analysis:  Physical  properties  are  the  predominant  method  of 
writing  specifications  (81%)  .  Performance  specifications  are 
used  less  than  20%  of  the  time. 
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Topic  B  Conclusion:  Clearly  stated  technical  performance 

requirements  are  not  used  as  a  deliberate  part  of  the  design 

V 

process.  Translation  of  requirements  into  technical  terms 
appears  to  be  a  byproduct  of  the  design  process. 

TOPIC  C:  The  relative  difficulty  of  obtaining  information 
inside  and  outside  the  engineer's  immediate  area  of  expertise 
during  the  design  process. 

C.l.  Is  information  outside  of  the  immediate  area  of 
expertise  more  difficult  to  identify  and  obtain? 

(Questions  22,11) 

Q22:  [How  difficult  is  it  to  obtain  information  from  dif¬ 
ferent  human  sources?] 

On  a  scale  of  0  to  4,  customers  (2.5),  manufacturing  (2.3), 
management  (2. 2), and  marketing  (2.1)  were  the  top  four  cited 
in  order  of  difficulty.  Other  engineers  (1.4),  vendors  (1.3) 
and  quality  assurance  (1.2) ,  all  relatively  close  to  the  area 
of  expertise,  were  cited  as  the  least  difficult  sources  from 
which  to  obtain  information. 
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Qll:  [Why  is  it  difficult  to  obtain  information  on  various 
topics?] 

Table  18  consolidates  the  raw  scores  of  question  11.  The 
topics  listed  across  the  top  are  arranged  in  general  order 
of  proximity  to  the  area  of  expertise,  with  the  closest  topic 
on  the  left  and  the  most  distant  topic  on  the  right. 
Response  "j"  (does  not  apply,  information  is  easily  obtained) 
decreases  from  left  to  right,  while  the  sums  of  responses  "a" 
through  "i"  (all  reasons  for  difficulty)  increase.  The  only 
exception  to  the  trend  is  the  topic  laws  and  regulations. 

Analysis:  Correlating  questions  11  and  22,  marketing  informa¬ 
tion  is  the  most  difficult  to  obtain,  followed  closely  by 
information  about  the  product  manufacturing  process. 

C.2.  Is  an  increase  in  difficulty  due  to: 

a.  Lack  of  subject  knowledge,  either  background  under¬ 
standing  or  knowledge  specific  to  the  design  project? 

b.  Time  and  cost  constraints  of  the  design  project? 

c.  Lack  of  common  language? 


(Question  11) 
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Qll:  [Why  is  it  difficult  to  obtain  information  on  various 
topics?] 

Analysis:  Referring  to  Table  18,  lack  of  time  and  nonavaila¬ 
bility  of  information  were  cited  as  strong  reasons  for  having 
difficulty  obtaining  information  about  both  marketing  and 
product.  For  marketing,  two  additional  reasons  were  (1) 
knowing  what  to  ask  and  (2)  someone  being  uncooperative. 
Knowing  whom  to  ask  increased  in  difficulty  for  nontechnical 
issues  (distribution,  marketing  and  legal) . 
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Table  18.  Reasons  for  Difficulty  in  Obtaining  Information 


\  TOPICS 
REASONS  \ 

_ 

CLOSE  TO  AREA  FAR  FROM  AREA 

OF  EXPERTISE  '  OF  EXPERTISE 

— 

TECHNICAL 

_  _ _ _ _ _ 

NON-TECHNICAL 

Packaging 

Material 

Packaging 

Operation 

Product 

Manufacturing 

Process 

Distribution 
of  Product 

Marketing 

— 

Laws 

Regs 

a 

— 

Lack  of 

T  ime 

8 

5 

10 

3 

5 

10 

5 

b 

Too 

Costly 

2 

3 

■ 

1 

1 

6 

0 

c 

Knowing 

Whom  to  Ask 

5 

— 

5 

4 

11 

8 

11 

d 

Ask  the  Right 
Question 

8 

— 

6 

5 

7 

9 

12 

5 

e 

Understanding 
The  Answer 

2 

7 

4 

5 

4 

4 

7 

f 

Information 
Not  Available 

5 

5 

5 

11 

15 

7 

g 

Do  Not  Use 
Same  Language 

1 

2 

n 

5 

1 

6 

4 

h 

Someone  is 
Uncooperative 

1 

5 

5 

2 

2 

9 

0 

i 

Other 

Reasons 

3 

1 

i 

3 

2 

1 

0 

|Sun  of  Reasons  j 
|For  Difficulty  |  35 

_ 1 _ 

38 

52 

35 

46 

71 

39 

li 

U 

I 

Not  Difficult]  14 

i 

13 

10 

14 

10 

6 

16 

j 
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C.3.  With  whom  do  engineers  most  often  communicate  to  obtain 
knowledge  and  information?  (Question  9,12,13,14) 

Q9 :  [With  whom  are  design  requirements  discussed?] 

Q12 :  [How  frequently  is  information  sought  in  various  areas?] 
Q13:  [With  whom  are  technical  values  of  requirements  dis¬ 
cussed?] 

Q14:  [Who  contributes  to  developing  design  requirements?] 

Analysis:  Correlating  responses  to  the  four  questions,  the 
information  is  most  often  sought  from  production,  product  or 
manufacturing  engineers,  and  marketing,  in  descending  order. 

C.4.  What  information  and  knowledge  is  most  important? 
(Questions  6,7,12,16,18,19} 

Q6:  [How  important  is  information  in  various  areas?] 

Q7:  [What  information  is  sought  first  in  a  design  problem?] 
Information  about  the  product  is  most  important,  followed  by 
distribution,  the  packaging  materials,  and  production  quan¬ 
tities.  Marketing  also  received  strong  mention,  ranked  fifth 
in  question  6,  but  not  listed  in  question  7. 
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Q12:  [How  frequently  is  information  sought  in  various  areas?] 

Although  a  direct  correlation  of  topics  cannot  be  made, 
frequency  of  information  sought  supports  the  same  five 
categories  found  in  question  6. 

Q16:  [What  elements  are  included  in  design  requirements?] 
Q18:  [What  internal  factors  influence  design  requirements?] 
Q19:  [What  external  factors  influence  design  requirements?] 

Information  about  the  product,  packaging,  and  production 
received  strong  mention  as  information  needed  to  develop 
requirements.  Marketing  was  not  listed  among  the  options  for 
these  three  questions. 

Analysis:  Questions  12,  16,  18  and  19  correlate  the  findings 
of  Questions  6  and  7.  The  most  important  information  and 
knowledge  is  about  the  product,  the  packaging  materials, 
product  distribution,  production  rates  or  quantities,  and 
product  marketing. 

Topic  C  Conclusion:  Information  and  knowledge  outside  the 
immediate  area  of  expertise  is  more  difficult  to  obtain. 
Information  about  marketing  is  the  one  area  which  combines 
a  high  level  of  importance  and  high  relative  difficulty  to 
obtain  information. 
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APPENDIX  C 


C.l  Instructions  for  Participants 

In  this  exercise,  you  will  be  presented  a  product  that 
requires  packaging.  Your  requirement  is  to  begin  the  process 
that  will  lead  to  the  development  of  feasible  packaging 
concepts.  You  should  work  toward  concepts  that  can  be  easily 
implemented  in  a  timely  manner  under  ordinary  budget  restric¬ 
tions.  You  should  proceed  as  normal,  except  that  you  should 
think  aloud  as  you  work.  You  should  talk  constantly  from  the 
time  you  receive  the  product  until  the  exercise  is  completed. 
Do  not  plan  out  what  you  are  going  to  say  or  attempt  to 
explain  your  thoughts.  Simply  act  as  if  you  are  talking  aloud 
to  yourself. 

Use  the  paper  and  pens  provided  to  sketch  or  make  notes. 
The  sheets  of  paper  are  numbered.  Try  to  make  your  notes  and 
sketches  sequential  from  top  to  bottom  on  each  page,  but  feel 
free  to  return  to  previous  notes  and  sketches  at  any  time. 

There  will  be  times  when  your  thoughts  are  nonverbal.  In 
those  cases,  just  say  something  like  "I  am  remember ing ... " 
or  "I  am  visualizing...."  You  will  not  be  interrupted,  other 
than  to  be  reminded  to  think  aloud. 

The  main  focus  of  this  exercise  is  on  what  information  you 
need  when,  and  where  you  get  it.  You  may  use  any  references 
you  desire,  or  you  may  just  say  aloud  what  you  would  do.  If 
you  would  normally  contact  someone,  just  state  who  and  for 
what  purpose.  If  the  information  is  known,  I  will  provide 
it  in  response. 

The  emphasis  is  on  your  experienced  approach  and  way  of 
thinking,  not  on  the  resulting  packaging  concepts.  Try  to 
work  as  you  normally  would,  but  remember  to  think  aloud. 
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C.2  Product  Descriptions 

Product  Description 

General:  A  sterile,  disposable  double-tipped  cotton  swab 

stick.  One  tip  is  saturated  with  a  alcohol  based  antiseptic 
solution  for  cleaning  an  infected  area.  The  other  tip  is  dry 
for  removing  excess  liquid  after  cleaning. 

Available  information  (not  provided  to  the  participant) : 

1.  User  requirements: 

a.  Maintain  sterility  of  both  tips. 

b.  Confine  liquid  to  saturated  tip. 

c.  Easy  to  open. 

d.  User  can  expose  each  tip  separately. 

e.  Should  be  able  to  reclose  the  wetted  tip  prior  to 
disposal . 

f.  User  can  easily  identify  which  tip  has  antiseptic 
solution  before  opening. 

g.  Should  be  able  to  minimize  exposure  of  a  contaminated 
(e.g.,  bloodstained)  swab  to  the  environment  during  disposal. 

2.  Shaft 

a.  Length:  5+0.05  inches 

b.  Diameter:  0.10+0.01  inches 

c.  Non-porous,  non-absorbing  fiber 

d.  Flexible  enough  to  withstand  180  degree  bend  when  held 


at  ends. 
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3 .  Tips 

a.  Spun  cotton 

b.  Wrapped  and  clipped 

c.  Volume:  0.25+0.02  inch  diameter,  0.5+0.02  inch  length 

4 .  Solution 

a.  Colorless  antiseptic  in  80%  alcohol  solution 

b.  Sensitive  to  UV  light 

c.  Not  sensitive  to  oxidation 

d.  Initial  volume  0.5+0.01  ml 

e.  Allowable  loss:  20%  by  weight  in  one  year 

5.  Distribution  system 

a.  Desired  shelf  life:  one  year 

b.  Method  of  transportation:  common  carrier  truck 

c.  Warehousing  and  storage:  not  controlled  by  company 

d.  Storage  conditions:  worldwide  climatic  conditions, 
covered  from  direct  exposure  to  weather 

e.  Desired  marketing  container  size 

(1)  10  per  unit  of  issue 

(2)  24  units  per  carton 

6.  Production 

a.  Rate  of  production:  300  per  minute 

b.  Swabs  supplied  in  bulk  from  vendor 

c.  Machine  available  for  dipping  swab  into  solution 

d.  Space  and  funds  are  available  to  modify  production  line 
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PRODUCT  DESCRIPTION 

General:  The  product  to  be  packaged  is  a  single  serving, 

microwavable,  shelf-stable  pizza.  The  product  consists  of 
a  baked  crust,  a  tomato  based  sauce  and  a  processed  cheese 
topping.  Consumer  must  transfer  shaped  slice  of  cheese  to 
crust,  spread  tomato  sauce  over  the  cheese  and  bake  for  90- 
120  seconds  in  microwave  on  maximum  power  setting. 

Available  information  (not  provided  to  participant) : 

1 .  Crust 

a.  Size  in  square  inches:  30 

b.  Weight:  6  oz  (170g) 

c.  Ingredients:  enriched  wheat  flour, partially  hydrogena¬ 
ted  vegetable  oil  shortening  (soybean  or  cottonseed) ,  water, 
yeast,  dextrose,  leavening  (sodium  aluminum  phosphate  and 
sodium  bicarbonate) ,  salt. 

d.  Preparation  method:  baked  in  radiant  oven 

e.  Sensitive  to 

(1)  moisture  pickup  and  dehydration 

(a)  water  content  is  37% 

(b)  water  content  tolerance  +/-  0.5% 

(2)  oxygen  concentration  must  be  less  than  3% 

(3)  sensitive  to  UV  light 

(4)  shock  -  not  sensitive 

(5)  static  load  -  cannot  bear  any  load 

(6)  vibration  -  not  critical 
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f .  Mixed  and  baked  at  425  degrees  Fahrenheit  for  12  minutes 
in-house 

2.  Tomato  based  sauce 

a.  Volume  -  6  oz.  (226g) 

b.  Ingredients  -  water,  tomato  paste,  soybean  oil,  modified 
food  starch,  salt,  oregano,  natural  flavorings 

c.  Prepared  in-house 

(1)  tomatoes  blanched 

(2)  food  starch  dissolved  in  cool  water 

(3)  warm  water  added  to  tomatoes,  then  food  starch  and 
dry  ingredients. 

(4)  mixture  heated  to  180  degrees  Fahrenheit  (maximum 
allowable  temperature) 

d.  viscosity  (range  is  similar  to  molasses) : 

(1)  6000  centipoise  at  70  degrees  Fahrenheit 

(2)  1800  centipoise  at  100  degrees  Fahrenheit 

(3)  900  centipoise  at  120  degrees  Fahrenheit 

(4)  300  centipoise  at  150  degrees  Fahrenheit 

e.  Sensitive  to 

(1)  no  oxygen  permissible 

(2)  UV  light  sensitive 

f.  Water  content:  68%  (Solids:  32%) 

3.  Cheese  topping 


a.  Volume  -  1.25  oz  (28g) 

b.  Processed  and  sliced 
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c.  Shaped  to  fit  crust  so  it  can  be  transferred  in  one 
piece 

d.  Solid  consistency 

e.  Sensitive  to 

(1)  moisture 

(a)  water  content:  32% 

(b)  water  content  tolerance:  +/-  0.1% 

(2)  oxygen  concentration  must  be  less  than  3% 

f.  Vendor  supplied  in  size,  shape  and  packaging  specified 

4.  Distribution 

a.  Desired  shelf  life:  60  days  from  production 

b.  Method  of  transportation  -  common  carrier  truck 

c.  Intermediate  and  final  warehousing 

(1)  geographic  warehouses  in  major  sales  areas 

(2)  local  warehousing  in  major  metropolitan  areas 

d.  Display  requirements  -  maximum  frontal  exposure  area 

e.  Desired  marketing  quantity  per  container  -  one  dozen  per 
sales  carton 

5.  Production  process  and  equipment 

a.  General  process 

(1)  mixing  vat  and  shaper  for  raw  crust 

(2)  conveyer  belt  radiant  oven  to  bake  crust 

(3)  mixing  and  blanching  vat  for  tomato  sauce 

b.  Production  rate  -  600  per  hour 

c.  Production  volume  -  six  month  test  market 
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6.  Corporation 

a.  Established  national  food  distributor 

b.  Six  regional  production  facilities 

c.  Willing  to  invest  in  equipment 
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PRODUCT  DESCRIPTION 

The  product  to  be  packaged  is  a  high  definition  television 
(HDTV)  with  a  30  inch  diagonal  viewing  screen.  The  televis¬ 
ion  is  portable  (no  console)  with  a  wireless  remote  control. 
The  product  is  a  new  introduction  to  the  market.  Manufac¬ 
turer's  Suggested  Retail  Price  is  $3700  including  delivery 
and  installation  by  the  local  retailer. 

Available  information  (not  provided  to  participant) : 

1.  Product  attributes 

a.  Weight:  85  pounds 

b.  Dimensions:  36"  long  x  32"  high  x  26"  deep 

c.  Electrical: 

(1)  120/240  volt,  50/60  Hz ,  40  watts 

(2)  10  foot  electrical  cord  wrapped  on  stakes  on  back 

side 

(3)  all  printed  wiring  assemblies  mounted  on  rubber 
shock  absorbers 

d.  Remote  control 

(1)  0.3  pounds 

(2)  2"  x  6"  x  0.5" 

(3)  two  size  AAA  1.5  volt  batteries  included  (not  in¬ 
stalled) 

(4)  vendor  supplied 

e.  Known  Sensitivities 

(1)  shock  limit:  40G  in  any  primary  direction 
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(2)  temperature  Limits:  -35  to  135  degrees  Fahrenheit 

(3)  moisture  Limit:  Relative  humidity  over  45%  at  80 
degrees  Fahrenheit  for  3  days 

(4)  compression  load  limits  on  television  case: 

(a)  250  pounds  from  top  or  bottom 

(b)  150  pounds  from  left  or  right  end 

(c)  50  pounds  from  front  or  back 

(5)  critical  natural  frequencies: 

(a)  7.5  Hz 

(b)  47  Hz 

(c)  354  Hz 

(6)  nlz  -.s  is  sensitive  to  scratching 
2 .  Production 

a.  Assembly  line 

(1)  bench  assembly 

(2)  variable  rate  handling  system  (Power/free  capabil¬ 
ity) 

(3)  36  stations  including  quality  assurance 

b.  Production  rate:  10  -  12  per  hour 

c.  Production  forecast:  50,000  -  75,000  units  before  first 
major  modification 
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3.  Distribution 

a.  International 

b.  Ship,  rail,  truck 

c.  Shipping  time:  6+/-1  days  domestic,  27+/ -3  days  interna¬ 
tional 

d.  Warehousing:  controlled  by  shippers 

e.  Sales  unit:  individual 

f.  Package  is  not  used  for  product  display 

g.  One  year  on-site  service  contract  included  in  sales 
price. 

4 .  Corporation 

a.  No  internal  packaging  department 

b.  Established  name  brand 

c.  One  assembly  plant  for  this  product 

d.  Corporate  goal  of  minimal  equipment  investments  for  next 


24  months 
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C.3  Edited  Protocol  of  Expert  A 

Note:  Bold  typeface  indicates  Researcher  input.  Normal 
typeface  is  test  subject  protocol. 

1  At  this  point,  the  customer  is  not  intimately  involved  with 

2  packaging,  they  don't  pick  it  up  from  the  store.  The 

3  effect  of  using  the  package  to  communicate  anything  about 

4  the  company  or  the  product  to  the  customer  is  not  as  big 

5  a  factor  as  it  would  if  it  were  a  retail  product  and  sat 

6  on  a  dealer  shelf  in  some  store  location.  So  as  far  as 

7  that  aspect  of  it,  we  don't  have  to  worry  about  the  box  as 

8  far  as  it  concerns  any  communication  on  the  box.  I  am 

9  assuming  that  it'll  be  bulk  packed  to  the  dealer,  but... 

10  as  a  lot  of  televisions  are.  But  with  this  high  end 

11  product,  I  doubt  that  the  dealer  will  order  more  than  a 

12  couple  at  a  time.  So  with  that,  I  will  look  to  design 

13  packaging  for  this  thing  as  an  individually  packaged  unit, 

14  as  opposed  to  a  bulk  packaged  unit.  Perhaps  the  first 

15  basic  approach  is  that  the  package  should  allow  the  product 

16  to  be  shipped  on  its  own  to  the  environment.  30"  diagonal 

17  screen — I  have  problems  trying  to  imagine  that  dimen- 

18  sion...I'd  like  to  find  out  what  the  physical  dimensions 

19  of  the  product  are,  dimensions  and  weight. 
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2  0  It's  3  6*'  long  by  32"  high,  and  2  6"  deep.  And  the  tele- 

21  vision  weighs  85  pounds. 

22  I  am  off  to  get  a  tape  measure  so  I  can  get  some  visualiza- 

23  tion.  I  would  like  to  get  a  product.  Is  the  product 

24  already  into  production  where  I  can  get  a  production  unit? 

25  Not  yet.  Still  in  the  design  stage. 

26  Okay.  Can  I  get  a  prototype  unit? 

27  Not  this  hour,  but  we  could  have  one  sent. 

28  Okay,  so  we  can't  have  a  model  or  an  actual  unit.  Is  the 

29  mechanical  design  complete  on  this? 

30  Yes. 

31  What  is  first  year  sales  expectations  for  this  product? 

32  The  goal  is  between  50  and  75  thousand  over  the  first  year. 

33  So,  fairly  low  volume ... [using  tape  measure  to  estimate 

34  size]  Dimensioning  the  size  and  the  weight,  the  package 

35  will  have  to  be,.,  it  can't  be  handled  by  one  person.  Be 
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36  either  mechanically  packaged,  or  packaged  with  mechanical 

37  handling,  or  by  two-men  carry.  Since  we're  delivering  it 

38  to  retail  locations,  a  lot  of  retailers  do  not  have 

39  sophisticated  mechanical  delivery  system,  and  they  have 

40  commonly .. .Well ,  how  many  distributors  or  retailers  will 

41  be... Will  there  be  a  lot  of  them  or  is  there  a  select  few 

42  that  are . . . ? 

43  It's  going  to  be  distributed  both  domestic  and  internation- 

44  al.  So  it's  going  to  be  quite  a  few  distributors  and 

45  retailers. 

46  Okay.  It's  safer  at  this  time  not  to  make  the  assumption 

47  that  the  final  recipient  of  the  packaged  product  will  not 

48  have,  probably  won't  have  any  sophisticated  means  of 

49  mechanical  handling  of  the  product.  Even,  obviously  a 

50  forklift.  I  doubt  that  would  be  usable.  Maybe  even  a 

51  pallet  jack,  especially  if  it's  being  delivered  to  the 

52  customer.  Now,  is  this  going  to  be  a  home-used  product? 

53  Yes,  a  home-used  product. 

54  So  I  doubt  that  even  that,  maybe  a  dolly  would  be  all  he'd 

55  have.  So  I  have  to  facilitate  for  at  least  a  two-man  carry. 

56  You  said  I  can  have  a  uni-?  Can  I  get  three  units? 
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57  What  do  you  need  them  for? 

58  For  fragility  testing. 

59  That's  been  run  on  a  prototype.  We  have  some  test  data 

60  available. 

61  okay,  what  is  fragility  testing  for  shock  and  vibrations? 

62  Shock  has  a  40g  limit  in  any  primary  direction. 

63  okay.  What  about  fragility  in  critical  frequency? 

64  critical  frequency,  three  were  identified:  7.5  hertz,  47 

65  hertz,  and  354  hertz. 

66  What  are  the  amplitudes  of  those? 

67  I  don't  have  amplitudes  on  those.  Those  are  the  natural 

68  frequencies. 

69  In  the  fragility  testing,  did  the  product  damage  when  held 

70  at  any  of  these  frequencies? 


71  Yes,  they  were  deunaged  at  all  three  of  those  frequencies. 
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72  if  held  there. 

73  Okay.  What  input? 

74  I  don't  have  the  input  available. 

75  We're  going  to  assume  an  input  of  half  a  g.  These  are 

76  assumed  to  be  sine.  That  was  based  on  the  sine  sweep? 

77  Yes 

78  Whew,  7.5  hertz,  that's  awful  low.  You  said  40g  in  any 

79  direction,  that's  a  critical  acceleration? 

80  Yes,  the  limit. 

81  Okay.  Is  that  based  on  a  square  wave? 

82  Impact  loading  drop  test 

83  Okay,  but  is  that  based  on  square  weight  testing,  or  a  half 

84  sine?  I'm  trying  to  see  whether  or  not  it 's. .. [sketched 

85  wave  form  of  input] 


86  Assume  square  wave. 
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87  Okay.  So  it  is  critical  ....  Based  on  the  distribution 

88  cycle,  what  kind  of  distribution  handling  does  it  receive? 

89  . .  would  tell  me  what  kind  of  qualification  testing  or  drop 

90  testing. . . ? 

91  It's  going  to  be  shipped  by  combination  of  truck,  rail  and 

92  ship  for  international.  And  the  warehousing  is  controlled 

93  by  the  shippers.  We  use  common  carriers,  we  don't  control 

94  it,  so  we  cannot  anticipate  what  will  happen  in  warehouse 

95  ing. 

96  What  was  the  critical  velocity  change  in  this  fragility 

97  testing? 

98  I'm  sorry,  they  didn't  give  that  to  me. 

99  What  was  the  velocity  change  used  in  the  fragility  testing 

100  to  achieve  that  40g? 

101  I  don't  have  that  either. 

102  Okay,  I'm  going  to  make  an  assumption  that  it  should  have 

103  been  done  from  something  equivalent  to  about  a  19"  drop 

104  height,  based  on  the  product's  weight  and  dimensions  ... 
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105  [calculations  made]  ...  so  I  am  assuming  it  is  117  inches 

106  per  second  velocity  change  when  that  was  done.  You  don't 

107  know  what  the  amplitude  of  the  vibration  is  at  the  7.5 

108  hertz,  or  any  of  these  frequencies? 

109  No. 

110  It's  just  identified.  I'm  quite  concerned  about  the  7.5 

111  hertz  natural  frequency,  because  that's  a  very  common 

112  frequency  in  the  distribution  environment,  especially  with 

113  trucks.  Truck  transportation  and  some  ship  transportation. 

114  I'm  assuming  that  I  get  a  fairly  significant  amplitude  and 

115  say  that  I'm  going  to  get  about  a  6g  response  at  the  7.5 

116  hertz,  which  it  says  it's  given  at  displacement,  whatever 

117  that  is,  it  gives  me  some  displacement,  almost  2"?  I 

118  wonder  what  that  is.  You  don't  know  what  breaks? 

119  Internal  parts,  I  don't  have  a  breakdown  of  that. 

120  I'm  trying  to  think  of  inside  a  TV,  something  that  way,  it 

121  could  be  something  on  the  processor  board,  could  be  a 

122  transformer  on  the  processor  board  with  a  weak. . [unintel- 

123  ligible].  Could  be  the  mounting  of  the  glass.  This  is 

124  a  glass  tube  TV,  is  it? 
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125  That's  correct. 

126  [Long  pause]  I'm  trying  to  visualize  what  might  have  been 

127  happening  in  vibration  testing.  One  reason  I  focus  a  lot 

128  on  the  vibration  testing  is  that  in  a  distribution  environ- 

129  ment,  shock  is  a  probability  function.  Shock  occurs  when 

130  the  vehicle  hits  a  bump  or  the  package  shifts  and  drops, 

131  or  someone  picks  up  the  package  and  drops  it.  And  again, 

132  it's  based  on  probability  of  these  incidents  occurring. 

133  Vibration  is  not  a  probability,  it's  100%  sure  if  it  gets 

134  in  a  truck.  I  try  to  primarily  look  at  that  aspect, 

135  because  that's  what  has  a  tendency  to  occur.  I'm  trying 

136  to  figure  out  what  kind  of  assumption  here  I'm  going  to 

137  make  as  far  as  what  the  7.5  hertz  situation  is.  This  being 

138  delivered  by  the  local  retailer,  it's  possible  that  minor 

139  things  could  be  done,  and  again,  it  depends  on  the  com- 

140  pany's  policy.  Our  policy  here  is  that  we  have  a  "plug  in 

141  and  play"  philosophy.  When  a  customer  gets  it,  he  should 

142  be  able  to  plug  it  in  and  starts  running  it  from  there. 

143  Because  of  that,  we  do  not  do  things  like  shipping  plugs 

144  or  reinforcement  devices  that  could  help  the  product  in  the 

145  distribution  environment,  but  once  it  gets  to  the  cus- 

146  tomer's  location,  it  no  longer  needs  any  vibration-type 

147  devices  on  it,  and  it  can  be  taken  out  and  discarded  by 

148  whoever  installs  it.  That's  why  I'm  trying  to  ponder  what 
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149  the  assumption  would  be  about  what  happens  at  7.5  hertz. 

150  I'm  having  problems  trying  to  switch  from  the  way  we 

151  currently  design  our  products  to  making  an  assumption. 

152  Let's  assume  that  it's  a  power  supply  and  that  there  is  an 

153  access  door  that  whoever  does  the  installation  can  reach 

154  inside  and  take  a  piece  of  corrugated  block  material  that 

155  would  support  the  power  supply  through  shipment  so  that  the 

156  7.5  hertz  is  not  a  problem  and  that  kind  of  damage  is  taken 

157  care  of.  That's  how  I'm  going  to  address  that  issue,  is 

158  that  we'll  use  a  shipment  plug  to  brace  the  power  supply 

159  and  that  can  be  discarded.  And  that  it  only  adds  minimal 

160  cost  to  the  product.  Maybe  less  than  2  0  cents  to  the 

161  packaging,  rather  than  something  expensive  in  redesigning 

162  the  product.  That's  what  I  was  pondering  here.  So  the 

163  next  frequency  to  deal  with  is  47  hertz,  and  then  354 

164  hertz.  The  47  hertz  is  also  kind  of  an  interesting  range. 

165  There  you  need  to  do  some  uniqueness  in  choice  of  materials 

166  and  designing  the  profile  to  attenuate  that  input  at  47 

167  hertz.  With  a  range  of  materials  that  are  used,  a  polyure- 

168  thane  foam  material  would  attenuate  or  protect  against  47 

169  hertz  very  well.  However,  it  is  kind  of  cost  effective, 

170  or  not  very  cost  effective.  You  also  have — are  these 

171  warehoused? 

172  Yes,  the  company  does  not  control  warehousing. 
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173  Okay,  so  is  there  a  consideration  of  flammability?  If  the 

174  company  doesn't  control  warehousing,  then  they're  automati- 

175  cally  charged  anyway.  The  very  limitation  on  some  coun- 

176  tries  such  as  in  Europe,  there's  requirement  for  packaging 

177  material  be  of  inflammable  material.  And  with  our  situa- 

178  tion,  we  don't  address  that  issue.  We've  seen  very  little 

179  to  be  effected  by  it,  and  paying  the  penalty  is  automati- 

180  cally  assumed.  And  for  these  quantities  we  assume  that 

181  that's  the  same  situation  here.  We're  not  going  to  regard 

182  the  flammability  issue  to  be  a  factor.  In  looking  again 

183  at  the  47  hertz,  a  styrene  would  be  a  very  good,.,  is  a 

184  very  common  cushioning  product  for  use  in  televisions  set, 

185  especially  the  cheaper  television  sets.  However,  it  has 

186  a  big  problem  with  vibration.  It's  very  difficult  to 

187  design  for  vibration.  And  it  normally  gives  you  a  lot 

188  higher  frequencies,  normally  to  be  able  to  get  a  good  shock 

189  response  in  a  design,.,  common  frequency  responses  for 

190  styrene  is  like  around  80  to  70  Hertz  which  would  put  you 

191  up  to  have  a  problem  with  this  47  Hertz  situation.  Also, 

192  a  secondary  issue  is  that  environmentally  the  consumers 

193  are  getting  educated  to  the  point  to  where  it's  a  product 

194  that  primarily  uses  CFCs  to  expand  and  people  that  are 

195  environmentalists  wouldn't  want  to  support  a  kind  of 

196  product  that  uses  that.  There  are  also  landfill  dis- 
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197  posability  issues  to  where  dealers,  especially  the  north- 

198  eastern  United  States  who  I  have  known  to  refuse  to  order 

199  a  product  based  on  the  fact  that  it's  packaged  in  styrene, 

200  because  they  are  penalized  by  the  volume  and  amount  of 

201  cushioning  or  packaging  that  they  have  to  throw  away  into 

202  the  landfill.  So,  on  that  basis,  I  would  kind  of  steer 

203  away  from  styrene.  Based  on  those  concerns.  That  puts  me 

204  into  the  range  of  polyethylene  or  polypropylene  foams. 

205  There's  extruded  polyethylene  and  molded  polyethylene. 

206  Judging  from  the  size  of  this  product,  there  are  some  off- 

207  the-shelf  cushioning  that  are  premolded  into  corner  blocks 

208  that  can  be  bought,  it  doesn't  require  any  special  molding. 

209  So  at  this  point,  based  on  small  quantity  that  this  product 

210  has,  it'll  save  me  a  lot  of  tooling  costs,  because  molded 

211  tooling  costs  would  run  20  to  30  thousand  dollars.  And  I 

212  would  prefer  at  this  time  to  use  some  of  the  market  off- 

213  the-shelf  molded  cushions  made  out  of  some  kind  of  expanded 

214  polyethylene  or  expanded  polypropylene  beads.  That  would 

215  save  a  lot  of  design  and  engineering  time.  So  that  leaves 

216  me  basically  to  determine  how  thick  the  cushion  I  will 

217  need,  which  will  eventually  tell  me  what  size  box  I'll 

218  need.  From  the  drop  height  of  18  inches,  [calculations 

219  being  made]  requiring  it  to  protect  the  product  to  40g's 

220  using  a  cushion,  just  to  be  conservative,  that's  about  35% 

221  efficient  in  deflection.  I  would  need  1.3  inches  of 
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222  cushion.  Optimally  I  need  1.38.  Hopefully,  I  can  find, 

223  probably  what's  on  the  market  are  1.5"  or  2"  cushions, 

224  corner  cushions.  That's  what  I'll  try  to  design  with. 

225  Now,  you  gave  me  the  dimension  on  this  thing,  can  we  assume 

226  that  this  is  a  total  rectangular  TV,  or  is  it  sloped  in  the 

227  back? 

228  The  base  is  rectangular,  except  for  the  portion  where  the 

229  tube  sticks  out  there. 

230  Straight  back  like  you  drew  it  there,  except  in  the  middle 

231  of  the  back  there's  a  slight  projection  where  the  tube 

232  comes  out?  How  far  out  does  that  stick  out? 

233  Six  inches. 

234  You're  making  this  challenging!  Everything  else  is  square, 

235  though? 

236  That  is  correct,  just  all  square. 

237  Are  there  any  weak  areas  within  here  that  top  and  bottom 

238  where  the  plastic  is  awful  weak.  Can  I  assume  that  the 

239  corners  and  edges  are  the  strong  points? 
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240  There  are  no  known  weak  areas. 

241  No  known  weak  areas.  Again  a  6"  situation  sticking  out  the 

242  back.  That  changes  things.  I'm  going  to  have  to  ship  it 

243  on  its  face. 

244  Let's  say  we  can  get  rid  of  that  6",  just  keep  the  problem 

245  straightforward. 

246  Well,  I  think  I'm  going  to  ship  it  on  its  face  anyway. 

247  That's  the  most  stable  orientation  for  a  tube  because  of 

248  the  eg.  It  gives  you  a  very  low  eg.  Okay,  [referring  to 

249  sketch]  eg  is  right  here.  If  you  shipped  it  this  way, 

250  you'd  have  a  tendency  to  have  to  have  somebody  to  have  to 

251  pick  it  up  and  kind  of  rotate  on  them,  okay?  So,  for  all 

252  practical ...  it  looks  kind  of  weird,  though,  but  since  we're 

253  using  a  local  store  guy  to  deliver  it  to  the  customer,  he 

254  should  be  able  to...  shouldn't  have  any  problem  with  that, 

255  shipping  it  face  down.  Okay,  I'm  just  trying  to  get  an 

256  overview  of,  here  again  where  I  started  from.  I've  been 

257  challenged  here  with  a  new  product,  a  TV,  fairly  large, 

258  [using  tape  measure]  about  the  size  of  this  table,  isn't 

259  it?  Let's  see,  just  slightly  under  the  size  of  this  table. 

260  Gives  me  a  better  visual... So  if  I  load  it  face  down,  box 

261  is  32  by  36  [sketching] .  So  I  put  that  face  down  and  have 
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262  a  1.5  inch  cushion  here... 1.5  inch  here,  three  inches 

263  overall,  39  inches  to  the  top  of  the  box.  Are  there  any 

264  other  accessories  that  go  with  this,  or  is  it  just  one  TV? 

265  TV  and  a  remote  control. 

266  Okay.  No  manuals  or  operating  instructions? 

267  Yes,  the  manual  will  be  one  pamphlet  with  operating 

268  instructions  and  warranty  card. 

269  Manuals  and  pamphlet,  nothing  like  a  [notebook]  binder? 

270  No. 

271  Where's  the  manufacturing  location? 

272  The  manufacturer  is  going  to  be  located  in  Oklahoma. 

273  Because  of  the  size  of  the  box  and  the  size  of  the  product, 

274  and  considering  the,... right  now  I'm  just  kind  of  consider- 

275  ing  how  it  would  be  placed  and  how  it  would  be  cushioned. 

276  But  one  of  the  primary  things  is  how  the  customer,  or 

277  whoever  has  to  get  the  product  out  of  the  box  and  at  the 

278  same  time  be  able  to  ship  it  in  transportation.  It 
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279  sometimes  becomes  a  problem  when  the  packaging  is  so  tight 

280  that  when  it  gets  to  the  other  end,  the  customer  can't  get 

281  the  product  out,  it  makes  it  very  frustrating.  And  once 

282  you  start  getting  feedback,  that  means  that  people  are 

283  refusing  to  buy  the  product  because  of  the  packaging,  and 

284  that's  not  what  packaging  is  for.  And  that's  happened, 

285  I've  seen  that  happen.  [Took  a  break]  The  concept  I'm 

286  developing  here  is,  I'm  trying  to  make  it  where  it's  easy 

287  from  the  distribution,  when  it  moves  through  the  distribu- 

288  tion.  I'm  trying  to  understand  the  material  handling 

289  systems  that  might  be  available  where  this  thing  is  going. 

290  It  needs  to  be  able  to  accommodate  those.  A  lot  of  freight 

291  forwarding  places  would  prefer  to  have  something  that  they 

292  can  pick  up  with  a  forklift.  But  they  do  accommodate  to 

293  be  able  to  have  these  things  where  they  can  just  run  it 

294  through  the  conveyer  system.  This  type  of  product  with  its 

295  size,  I  doubt  that  it  will  go  through  any  kind  conveyer 

296  system  or  any  kind  of  distribution  environment.  So,  what 

297  I've  done  here  [sketch]  is  made  a  base  made  out  of  multi- 

298  layered  corrugated,  structurally  corrugated,  probably 

299  triple  walled,  with  some  kind  of  skids  on  it  that  would 

300  allow  a  forklift  to  go  under.  They  make  these  again  as  a 

301  standard  off-the-shelf  product  that  can  be  purchased  in 

302  preset  quantities.  This,  again,  would  limit  a  lot  of 

303  engineering  time  in  designing  these  pallets.  That,  with 
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304  minimal  modification,  could  be  done.  What  this  would  do 

305  is  give  you  a  little  cap  right  here,  and  then  you  can  glue 

306  these  corner  blocks  down  on  the  pallet  where  you  put  the 

307  unit  in.  The  top  part  is  a  sleeve  tha;-  fits  down  over  the 

308  unit.  And,  to  close  the  box,  we're  going  to  use  bands, 

309  plastic  bands  that  are  friction  welded.  Now  the  reason  for 

310  that  is  to  make  it  easier  on  the  other  end  as  far  as 

311  opening  it.  Friction-welded  bands  require  no  tooling,  just 

312  flip  it  over  and  peel  it,  but  it  holds  very  well.  And  it 

313  also  holds  as  well  as  metal  bands,  without  doing  a  lot  of 

314  cutting.  Probably  some  corner  boards  down  here.  We're 

315  going  to  put  hand  holds  right  here  so  it  allows  for  manual 

316  carry  for  those  situations,  in  places  where  you  don't  have 

317  mechanized  systems  to  go  do  that.  But  when  it  gets  to  the 

318  other  end,  to  unpack  it  all  you  basically  do  is  take  the 

319  top  cover  and  the  unit  is  exposed,  and  then  you  can  just 

320  tilt  it  from  that  and  move  it  into  location.  Okay,  and 

321  then  at  the  top,  if  it's  just  a  pamphlet,  I'll  probably 

322  tape  the  manual  to  the  side  of  the  container  with  a  non- 

323  residue  plastic  pouch.  Encase  the  product  in  a  plastic  bag 

324  to  prevent  any  dust  from  entering  into  the  system. 

325  Basically  just  to  protect  it  from  dust,  not  cosmetic 

326  aspect.  Okay,  next  thing  I  do  is  to  go  test  this  basic 

327  concept .... 
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APPENDIX  D 

D.l  Instructions  to  All  Participants 

You  will  be  given  a  description  of  what  I  need.  Write  a 
problem  statement  from  which  a  solution  could  be  developed. 
Do  not  solve  the  problem.  Write  your  statement  as  if  you 
will  give  it  to  another  qualified  mechanical  engineer  for 
developing  a  solution.  Write  the  statement  on  the  titled 
page. 

If  you  want  any  information,  write  down  your  question  on 
the  blank  pages  and  I  will  write  a  response.  You  do  not  have 
to  form  complete  sentences.  Please  number  your  questions 
(Ql,  Q2,  etc.)  and  I  will  number  the  responses  (Al,  A2,  etc). 


Control  Number 


Problem  Statement 
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D.2  Problem  Description 

PROBLEM  DESCRIPTION 

I  make  small  devices  which  require  two  coil  compression 
springs  with  equal  performance.  I  need  a  good  way  to 
determine  if  any  two  coil  springs  selected  at  random  from  a 
large  inventory  are  a  compatible  pair. 

Each  individual  spring  in  the  inventory  meets  manufactur¬ 
ing  tolerances.  Most  pairs  of  springs  are  compatible,  so  I 
need  to  detect  the  infrequent  pairs  that  are  not. 
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D.3  Guidelines  for  Treatment  Group 

Guide  to  Problem  Formulation 

In  writing  your  problem  statement,  I  want  to  challenge  you 
to  make  it  more  descriptive  of  both  the  problem  and  the 
environment  within  which  the  problem  must  be  solved.  Your 
statement  should  include  as  many  Technical  Performance 
Requirements  as  you  can  identify  at  this  point  in  the  design 
process. 

Technical  Performance  Requirements  describe  in  measurable 
or  verifiable  terms  the  performance  required  for  any  solution 
to  be  considered  acceptable.  Each  Technical  Performance 
Requirement  includes  four  elements: 

Identification  -  a  verbal  description  of  the  performance 
attribute  (e.g.,  strength,  speed,  cost,  durability,  weight) 

Conditions  -  delineation  of  the  environmental  conditions 
under  which  the  performance  is  required  (e.g.,  temperature, 
moisture  content,  time,  number  of  operators) 

Target  Value  -  the  optimum  performance  value,  that  value 
which  will  minimize  loss  or  maximize  gain  (e.g.,  32  miles 

per  gallon,  90  decibels,  63  Hertz) 

Allowable  Variance  -  the  permissible  deviation  from  the 
target  value  (e.g.,  +  2%,  no  more  than,  +  6  Hertz) 

It  is  important  that  you  describe  WHAT  performance  you 
need,  but  not  how  to  achieve  it.  You  should  not  restrict  the 
designer's  creative  ability,  but  should  instead  identify  the 
boundaries  within  which  to  be  creative.  If  you  specify  what 
levels  of  performance  are  required  for  any  solution,  the 
designer  should  be  able  to  determine  if  a  solution  is  within 
those  boundaries  and  also  when  the  solution  is  adequate 
enough  to  stop  designing. 

To  help  identify  performance  attributes,  think  and  ask 
questions  in  terms  of: 

TIME  (dates,  duration,  cycles,  etc.) 

DIMENSIONS  (distance,  areas,  weights,  etc.) 

VALUE  (precision,  cost,  quality,  reliability,  etc.) 

PHYSICAL  MATTER  (materials,  parts,  machines,  etc.) 

PEOPLE  (user,  manufacturer,  organizations,  etc.) 

Once  you  gather  your  information,  write  your  problem 
statement  to  include  the  Technical  Performance  Requirements 
you  have  identified. 


146 


D.4  Available  Information 

Spring  specifications  in  the  inventory: 

1.  Free  length:  range:  (4.9,  5.1); 

normally  distributed:  N(5,  0.03) 

2.  Spring  rates:  range  (lb/in) :  (0.98,  1.02); 

normally  distributed:  N(1.0,  .065) 

3.  Spring  diameters:  3  inches  (nominal)  +  0.1  inch 

4.  Material:  Music  wire,  uncoated 

5.  Wire  diameter:  10  gauge  nominal  (0.1  in.) 

6.  Number  of  turns:  15 

7.  Total  deflection:  3  inches  (nominal) 

8.  Weight:  4  ounces  (nominal)  ±  0.1  ounce 

Allowable  differences  for  a  pair  of  springs: 

1.  Difference  in  free  length:  0.1  in 

2.  Difference  in  spring  rate:  0.02  Ib/in 

3.  98%  probability  of  detecting  incompatibility  (Alpha=0. 02) 

4.  5%  probability  of  falsely  determining  a  compatible  pair 
to  be  incompatible  (Beta  =  0.5) 

System  information: 

1.  Load  range:  0-3  pounds,  static  loading,  non-cyclic,  no 
impact  loading. 

2.  Efficiency  requirement:  less  than  15  seconds  to  test 

3.  Weight  limit  for  tester:  8  pounds 

4.  Cost  limit  for  tester:  $50 

5.  Historically,  only  6%  of  pairs  tested  are  rejected  as  not 
compatible. 

6.  Number  of  testers  required:  5 

7.  Training  time  should  be  less  than  15  minutes. 

8.  Tester  should  be  portable,  requiring  no  fixed  installa¬ 
tion. 

9.  No  records  of  individual  test  are  required. 

10.  Must  be  available  for  use  in  4  months. 

11.  Male  and  female  assembly  line  operators. 

12.  500  -  700  pairs  of  springs  must  be  tested  per  day 

13.  Life  requirement  of  device  is  one  year. 

14.  Device  assembly  is  manual  operation. 

15.  Operating  environment  is  70  -  100  °F,  normal  atmospheric 
conditions. 
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D.S  Multiple  Attribute  Value  Model 


ATTRIBUTE 

WEIGHT 

UTILITY  SCALE 

SCORE 

UTILITY 

CLARITY 

1 

0  1 

SI 

SI 

SOLUTION 

IMPLICATION 

W2 

0  100 

S2 

W2XS2 

STRUCTURAL 

CONSIDERATION 

W3 

0  100 

S3 

W3xS3 

STAKEHOLDERS 

W4 

0  100 

S4 

W4xS4 

LEVEL  OF 

FOCUS 

W5 

0  100 

S5 

W5xS5 

PERFORMANCE 

IDENTIFICATION 

W6 

0  100 

S6 

W6XS6 

PERFORMANCE 

MEASURABILITY 

W7 

S7 

W7xS7 

QUALITY= 

SI  *  { E  Wi*Si } 
i~2 ,  3 , 4 , 5 , 6 , 7 

Legend : 

W;  Relative  weight  of  the  ith  attribute 

Sj Graders  score  of  the  problem  statement  with  respect  to  the 
ith  attribute 
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D.6  Instructions  for  Rating  Problem  Statements 

You  are  asked  to  rate  a  collection  of  problem  statements, 
each  of  which  was  developed  by  individuals  based  on  the 
situation  described  below: 


I  make  small  devices  which  require  two  coil 
compression  springs.  I  need  a  good  way  to 
determine  if  any  two  coil  springs  selected  at 
random  from  a  large  inventory  are  a  compatible 
pair. 

Each  individual  spring  in  the  inventory  meets 
manufacturing  specifications.  Most  pairs  of 
springs  are  compatible,  so  I  need  to  detect  the 
infrequent  pairs  that  are  not. 


Additional  information  pertinent  to  the  situation  is  listed 
on  the  next  page.  This  additional  information  was  not 
provided  to  the  participant,  but  was  used  to  respond  to 
specific  questions  asked  as  the  participant  developed  the 
problem  statement. 

Each  statement  will  first  be  assessed  to  determine  if  it 
is  written  with  sufficient  clarity  to  allow  you  to  evaluate 
it.  If  the  statement  is  unacceptable  due  to  lack  of  clarity, 
no  further  evaluation  is  required. 

Each  acceptable  statement  will  be  given  a  rating  between 
0  and  100  along  six  dimensions: 

-  Structural  Considerations 

-  Stakeholders 

-  Solution  Implication 

-  Level  of  Focus 

-  Performance  Identification 

-  Performance  Measurability 

Definitions  and  rating  scales  for  the  six  dimensions  are 
provided  on  subsequent  pages.  Please  read  the  definitions, 
then  reflect  on  what  these  dimensions  mean  to  the  specific 
situation  described  above.  It  may  be  helpful  to  note  your 
reflections  about  each  dimension  on  the  appropriate  defini¬ 
tion  page.  At  this  time,  you  are  establishing  your  personal 
rating  scheme  that  you  will  consistently  apply  in  evaluating 
each  statement. 
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The  six  dimensions  are  independent.  You  should  rate  the 
entire  set  of  statements  along  one  dimension  without  con¬ 
sideration  for  how  you  will  rate  it  along  the  other  five 
dimensions.  Each  scale  should  be  considered  a  continuum. 
You  should  rate  each  statement  using  the  end  points  and 
anchors  as  guides,  but  you  are  free  to  assign  any  value 
between  0  and  100. 

Once  you  have  established  your  rating  scheme,  please  follow 
this  sequence: 

1.  Read  each  statement  for  clarity.  If  a  statement  is 
unacceptable  due  to  lack  of  clarity,  mark  it  as  such  and 
remove  it  from  the  set  of  statements. 

2.  Shuffle  the  remaining  statements.  Randomly  select  one 
dimension  (out  of  the  six)  for  evaluation.  Examine  the  scale 
for  that  dimension,  then  rate  each  statement  along  that 
dimension. 

3.  Repeat  step  2  until  each  acceptable  statement  has  been 
rated  along  each  of  the  six  dimensions. 

Do  not  reward  or  penalize  statements  based  on  length, 
sentence  structure  and  grammar,  choice  of  words,  or  format. 
As  long  as  minimum  clarity  standards  have  been  met,  try  to 
evaluate  the  statements  solely  along  the  six  dimensions. 
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Available  Information 

Spring  specifications  in  the  inventory: 

1.  Free  length:  range:  (4.9,  5.1); 

normally  distributed:  N(5,  0.03) 

2.  Spring  rates:  range  (lb/in):  (0.98,  1.02); 

normally  distributed:  N(1.0,  .065) 

3.  Spring  diameters:  3  inches  (nominal)  +  0.1  inch 

4.  Material:  Music  wire,  uncoated 

5.  Wire  diameter:  10  gauge  nominal  (0.1  in.) 

6.  Number  of  turns:  15 

7.  Total  deflection:  3  inches  (nominal) 

8.  Weight:  4  ounces  (nominal)  ±0.1  ounce 

Allowable  differences  for  a  pair  of  springs: 

1.  Difference  in  free  length:  0.1  in 

2.  Difference  in  spring  rate:  0.02  lb/in 

3.  98%  probability  of  detecting  incompatibility  (Alpha=0.02) 

4.  5%  probability  of  falsely  determining  a  compatible  pair 
to  be  incompatible  (Beta  =  0.5) 

System  information: 

1.  Load  range:  0-3  pounds,  static  loading,  non-cyclic,  no 
impact  loading. 

2.  Efficiency  requirement:  less  than  15  seconds  to  test 

3.  Weight  limit  for  tester:  8  pounds 

4.  Cost  limit  for  tester:  $50 
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5.  Historically,  only  6%  of  pairs  tested  are  rejected  as  not 
compatible . 

6.  Number  of  testers  required:  5 

7.  Training  time  should  be  less  than  15  minutes. 

8.  Tester  should  be  portable,  requiring  no  fixed  installa¬ 
tion. 

9.  No  records  of  individual  test  are  required. 

10.  Must  be  available  for  use  in  4  months. 

11.  Male  and  female  assembly  line  operators. 

12.  500  -  700  pairs  of  springs  must  be  tested  per  day 

13.  Life  requirement  of  device  is  one  year. 

14.  Device  assembly  is  manual  operation. 

15.  Operating  environment  is  70  -  100  °F,  normal  atmospheric 
conditions . 
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Clarity  -  The  binary  assessment  of  whether  or  not  the 
problem  is  stated  in  a  coherent  way  and  in  understandable, 
non-j argon  terms  sufficient  to  allow  further  evaluation  of 
attributes . 


0 

NOT-CLEAR 

(Unacceptable) 


1 

CLEAR 

(Acceptable) 
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Solution  Implication  -  The  degree  to  which  the  statement 
suggests  solutions  or  ways  to  achieve  solutions  to  the 
problem.  The  more  a  statement  does  so,  the  less  favorable 
it  is  (i.e.,  this  dimension  has  a  negative  orientation). 


0 _ 100 

SOLVES  LEADS  OPEN-ENDED 

(Worst)  (Anchor)  (Best) 

SOLVES:  The  problem  statement  explicitly  states  a  solution 
or  a  set  (class)  of  solutions. 

LEADS:  The  problem  statement  suggest  or  leads  to  a  (set  of) 
solution(s)  or  a  way  of  arriving  at  one. 

OPEN-ENDED:  The  problem  statement  suggests  no  solutions  or 
ways  of  arriving  at  solutions;  it  merely  states  the  problem. 
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Structural  Considerations  -  The  degree  to  which  the  state¬ 
ment  reflects  consideration  for  all  relevant  and  unique 
aspects,  parts,  complexities  and  their  interrelationships  and 
intrarelationships  to  the  extent  that  can  be  seen  or  verified 
at  the  stage  of  problem  formulation. 


0 _ 100 

NONE  SOME  ALL 

(Worst)  (Anchor)  (Best) 

NONE:  The  problem  statement  reflects  no  consideration  of  the 
structure. 

SOME:  The  problem  statement  reflects  some  consideration  of 
the  structure. 

ALL:  The  problem  statement  reflects  full  consideration  of  the 
structure. 
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Stakeholders  -  The  degree  to  which  the  statement  reflects 
considerations  of  the  needs,  interests  and  power  of  all 
stakeholders,  i.e.,  parties  that  will  affect  or  be  affected, 
directly  or  indirectly,  by  the  solution  of  the  problem. 
Stakeholders  may  be  tangible  (e.g.,  an  interest  group)  or 
intangible  (e.g.,  the  "unborn"  generation),  may  have  high 
salience  (in  the  sense  of  being  motivated,  having  high  stake) 
or  low  salience,  and  may  be  vocal/outspoken  or  "invisible". 
(Note:  It  is  assumed  that  all  salient  stakeholders  are 
tangible,  but  not  all  tangible  stakeholders  are  salient.) 


0 _ 100 

NONE  SALIENT  TANGIBLE  ALL 

(Worst)  (Anchor  #1)  (Anchor  #2)  (Best) 

NONE:  The  problem  statement  reflects  that  no  stakeholder  has 
received  adequate  consideration. 

SALIENT:  The  problem  statement  reflects  consideration  of 
stakeholders  of  high  salience,  as  defined  above. 

TANGIBLE:  The  problem  statement  reflects  consideration  of 
stakeholders  that  are  tangible,  as  defined  above. 

ALL:  The  problem  statement  reflects  that  all  stakehold  ers 
have  received  adequate  consideration. 
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Level  of  Focus  -  The  degree  to  which  the  statement  defines 
as  high  a  level  or  as  broad  a  problem  as  possible. 


0 _ 100 

NARROW  SOMEWHAT  SOMEWHAT  BROAD 

NARROW  BROAD 

(Worst)  (Anchor  #1)  (Anchor  #2)  (Best) 

NARROW:  The  problem  is  too  narrowly  defined. 

SOMEWHAT  NARROW:  The  problem  is  somewhat  narrowly  defined. 

SOMEWHAT  BROAD:  The  problem  is  somewhat  broadly  defined. 

BROAD:  The  problem  is  defined  as  broadly  as  is  possible  and 
appropriate. 
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Performance  Identification  -  The  degree  to  which  the 
statement  defines  what  performance  is  required  to  satisfy 
the  identified  needs  of  the  problem. 


0 _ 100 

NOT  DEFINED  SOMEWHAT  DEFINED  DEFINED 

(Worst)  (Anchor)  (Best) 

NOT  DEFINED:  Required  performance  is  not  defined. 

SOMEWHAT  DEFINED:  Some  required  performance  is  defined. 

DEFINED:  All  required  performance  is  adequately  defined. 
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Performance  Measurability  -  The  degree  to  which  required 
performance  can  be  measured  or  verified. 


0 _ 100 

NOT  MEASURABLE  SOMEWHAT  MEASURABLE  MEASURABLE 

(Worst)  (Anchor)  (Best) 

NOT  MEASURABLE:  Required  performance  cannot  be  measured  or 
verified. 

SOMEWHAT  MEASURABLE:  Some  required  performance  can  be 
measured  or  verified  with  difficulty. 

MEASURABLE:  Required  performance  can  be  measured  or  verified 
using  existing  standards. 


Difficulty  could  be  due  to  technical,  cost,  time,  lack  of 
standards,  or  other  reasons. 
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APPENDIX  E 

E.l  Problem  Statements  from  Sample  Population 


Design  a  test  method  or  an  equipment  that  can  be  used  to 
determine  if  two  compression  springs  are  compatible  with  each 
other. 

The  test  method  or  equipment  must; 

a)  measure  the  tolerance  between  the  springs 
10  ^  ±0.1  in  (free  length) 
k  <  +0.02  lb/in  (spring  constant) 
e.g.  spring  1  spring  2  spring  1  spring  2 

1  4.9  5.1  4.9  5.0 


unacceptable  acceptable 

1.0  1.0  1.0  1.02 

b)  -  detect  incompatible  springs  98%  of  the  time 

-  not  falsely  detect  a  compatible  pair  as  incompatible 
more  than  5%  of  the  time 

c)  be  done  in  less  than  15  mins. 

d)  cost  less  than  $50 

Choice  of  test  method  is  unlimited.  Test  method  or  equipment 
must  be  easy  to  use  (user  friendly)  and  reliable  and  should 
not  change  the  property  or  performance  of  the  springs. 


CONTROL  #_23_ 

SOLUTION 

IMPLICATION 

LEVEL  OF 
FOCUS 

PERFORMANCE 

IDENTIFICATION 

GRADER 

CLARITY 

STRUCTURAL 

STAKEHOLDERS 

PERFORMANCE 

Acceptable _ 

Unacceptable _ 

CONSIDERATION 

VERIFIABILITY 
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Design  a  inexpensive  (<$50)  device  and/or  process  that  will 
be  ready  for  use  in  less  than  4  months  and  requires  less 
than  15  minutes  training,  to  determine  if  two  coil  springs 
are  of  equal  length  and  stiffness.  Springs  considered 
equal  in  length  are  <  .1  in.  different  in  length  and 
springs  considered  equal  in  stiffness  are  <  02  lb/in  dif¬ 
ferent  in  stiffness.  The  procedure  and/or  device  must 
require  less  than  15  sec.  to  make  this  determination,  and 
must  detect  an  unequal  pair  of  springs  98%  of  the  time  and 
must  not  declare  an  equal  pair  of  springs  unequal  more  than 
5%  of  the  time.  All  springs  will  have  a  nominal  weight  of 
10  ga,  Diam  of  3  inch  and  be  made  of  music  wire.  The 
maximum  and  minimum  length  of  the  springs  encountered  is 
5.01  and  4.99  inches  respectively  and  the  maximum  and 
minimum  stiffness  of  the  springs  encountered  is  1.02  lb/in 
and  .98  lb/ in  respectively. 


CONTROL  #_31_ 

SOLUTION 

IMPLICATION 

LEVEL  OF 
FOCUS 

PERFORMANCE 

IDENTIFICATION 

GRADER 

CLARITY 

STRUCTURAL 

STAKEHOLDERS 

PERFORMANCE 

Acceptable _ 

Unacceptable _ 

CONSIDERATION 

VERIFIABILITY 
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Pairs  of  springs  are  to  be  tested  to  detect  differences 
in  both  free  lengths  and  spring  constants.  The  free 
lengths  of  the  springs  range  from  4.9  to  5.1  inches  (mean  = 
5.0  inch)  and  spring  constant  -anges  from  0.98  to  1.02 
lbf/in  (mean  =  1.0  lbf/in) .  Two  springs  are  considered  to 
be  incompatible  if  the  free  lengths  differ  by  more  than  0.1 
inch  or  if  the  two  spring  constants  differ  by  more  than 
0.02  lbf/in  or  both  free  lengths  and  spring  constants 
differ  by  the  above  amounts. 

Develop  the  conceptual  methodology  for  detecting  incom¬ 
patible  pairs  of  springs  selected  at  random  from  a  large 
inventory.  The  methodology  is  to  be  implemented  into 
either  a  manual  process  or  an  automated  machine  which 
tests  the  springs.  Automation  is  preferred,  and  the  mech¬ 
anical  design  of  the  machinery  or  equipment  needed  is 
needed  also. 


STRUCTURAL  STAKEHOLDERS  PERFORMANCE 
CONSIDERATION  VERIFIABILITY 
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a)  Require  a  process  to  select  two  springs  with  equal 
spring  constant  'k'.  Could  check  by  applying  equal  force 
(compression)  on  each  spring  and  noting  deflection 

b)  Increase  tolerance  of  manufacturing  process  (don't  know 
if  this  is  problem  statement) 


CONTROL  #_24_ 

SOLUTION 

IMPLICATION 

LEVEL  OF 
FOCUS 

PERFORMANCE 

IDENTIFICATION 

GRADER 

CLARITY 

STRUCTURAL 

STAKEHOLDERS 

PERFORMANCE 

Acceptable _ 

Unacceptable _ 

CONSIDERATION 

VERIFIABILITY 
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Project:  We  require  a  testing  facility  that  will  assist  a 
worker  in  inspecting  free  spring  lengths  and  stiffness  by 
visual  inspection.  The  testing  device  must  be  quick  and 
reliable  within  the  specifications  below.  Testing  time 
should  be  minimized.  Hint  damping  in  the  facility  for 
determining  spring  stiffness. 

Spring  Dimensions:  Free  length:  1=5”  l^^.S”  lmh=5.1” 

/\  1  between  springs  <  0.1" 
Stiffness :  kn^l.O  lb/in  ^=0.98  lb/in 

kn^l.02  lb/ in 

Spring!-  k»pring2<  0.02  lbs/in 

Particulars:  Diameter:  3.0”  and  Weight:  4  oz  (light) 

Testing  Time:  Total  time  15  s.  Suggest  distribution: 

+*.  .  —  ^  c 

‘-inspection 

^compression-  i  0  S 

Testing  Conditions:  Dry  and  room  temp. 

Cost  <  $50  +  $5.00 
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The  compression  springs  which  are  made  of  music  wire  are  in 
small  devices.  In  order  to  determine  if  the  springs  are 
acceptable  for  application  a  compatibility  check  is  re¬ 
quired  to  be  made.  The  compatibility  check  is  to  determine 
if  any  two  coil  springs  selected  at  random  from  the  inven¬ 
tory  are  a  compatible  pair.  Wire  diameter  is  0.10  inch 
(nominal) ,  coil  diameter  is  3  inches  (nominal) ,  free  length 
varies  from  4.9  to  5.1  inches  with  a  mean  value  of  5.0 
inches.  The  vendor  has  assured  a  spring  constant  of  0.98  - 
1.02  lb/in  with  a  mean  value  of  1.00  lb/ inch.  Measuring 
instruments  and  load  applying  facilities  for  the  springs 
are  provided  in  the  shop. 
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There  is  a  large  inventory  of  springs.  A  performance  test 
needs  to  be  run  to  measure  the  compressive  resistance  of 
each  spring.  Springs  with  identical  compressive  strengths 
(stiffness)  need  to  be  paired  together.  The  dimensions  of 
the  springs  is  given  below. 

Spring:  Compressive  coil  spring  3  in.  Dia  x  5.0  Length  Norn. 

Material:  Music  wire 

Manuf.  Stiffness:  1.0  lb/in.  mean 

Any  spring  that  is  unable  to  be  paired  with  another 
(stiffness  does  not  match  any  other  spring)  is  to  be  set 
aside. 

Please  contact  me  if  there  are  any  questions  regarding  this 
matter. 
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A  device  is  required  which  will  test  2  springs  selected 
randomly  from  inventory  to  insure  compatibility  and  equal 
performance  in  our  product. 

The  test  will  be  done  prior  to  spring  installation  into  the 
product.  The  device  shall  indicate  a  pass/ fail  response 
when  two  springs  are  tested.  When  2  springs  indicate  a 
pass  response,  the  2  springs  will  be  kept  together.  If  2 
springs  indicate  a  fail  response,  the  springs  shall  be 
returned  to  inventory. 

Pass/fail  criteria: 

-  Spring  free  lengths  shall  not  differ  by  more  than  .01" 

-  Spring  constants  must  not  differ  by  more  than  .02  lb/in 
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Identification: 

Design  a  device  that  will  quickly  compare  the  free  length 
and  spring  constant  of  two  helical  compression  springs. 

Conditions: 

The  device  must  be  used  under  assembly  line  conditions  and 
must  be  capable  of  performing  the  required  task  within 
approximately  15  seconds  using  any  of  a  number  of  assembly 
line  workers. 

Target  value  and  acceptable  variance: 

The  two  springs  compared  shall  have  a  free  length  that  is 
5"  with  a  manufacturing  tolerance  of  +.1  but  the  two 
springs  must  have  a  free  length  within  0.1"  of  each  other. 
Also  the  two  springs  shall  have  a  spring  rate  of  1.0  lb/in 
with  a  manufacturing  tolerance  of  0.02  lb/in  but  the  rate 
of  the  two  springs  must  be  within  0.02  of  each  other. 
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A  method  and/or  design  is  required  to  determine  if  two 
coil  compression  springs,  chosen  at  random  from  a  large 
inventory,  are  compatible  enough  to  work  as  a  pair. 

Compatibility  is  a  function  of  two  manufacturing  vari¬ 
ables:  free  length  and  spring  rate.  The  difference  in  free 
length  between  two  springs  in  a  pair  must  not  exceed  0.1 
inch.  The  difference  in  spring  rate  between  two  springs  in 
a  pair  must  not  exceed  0.02  lb/in. 

The  parameters  of  the  springs  as  manufactured  are  as 
follows: 

Free  lengths:  5.0  inch  +  0.1  inch 
Spring  rate:  1.0  lb/ in  +  0.02  inch 
Outside  diameter:  3.0  in  +  0.1  in 
Inside  diameter:  2.8  in  +  0.1  in 
Weight:  4  oz  +  0.1  oz 

Stroke  (working  deflection) :  0  to  3  inches 

The  operating  conditions  for  the  springs  is  70  to  100  °F, 
normal  atmospheric  conditions. 
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Two  coil  springs  made  of  10  gage  music  wire  are  to  be 
selected  from  a  large  inventory.  All  springs  are  known  to 
have  4.9  -  5.1  in.  free  length  and  stiffness  k  =  .98  -  1.C2 
lb/in.  Springs  are  made  to  exercise  static  compression 
loads,  (0-3  lbf) . 

Need  a  device  that  will  do  the  following: 

1.  Allow  to  pick  two  springs  (not  more  than  .1"  difference 
in  free  length),  &  (not  more  than  .02  lb/in  difference 
between  their  stiffness) . 

2.  Device  is  to  be  manually  driven  (for  both  males  and 
females) . 

3.  Device  is  expected  to  live  for  1  year  &  to  perform  500- 
7  00  times  a  day  in  an  environment  of  70  -  90  °F.  Atmospher¬ 
ic  conditions  apply.  The  time  of  using  (operating)  the 
device  each  time  should  not  exceed  15  sec. 

4 .  Device  should  be  less  than  8  lbs  &  should  not  cost  more 
than  $50  to  produce  it. 

5.  Device  should  work  98%  of  the  time.  It  also  can  fail  to 
test  that  two  compatible  springs  are  incompatible  less  than 
5%  of  the  time. 

6.  Device  should  be  ready  in  4  months. 
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For  any  pair  of  randomly  selected  springs,  design  (devise) 
a  procedure  which  tests  the  difference  in  free  length  and 
stiffness.  If  the  difference  in  free  length  is  greater 
than  0.1  inches  or  if  the  difference  in  stiffness  is  great¬ 
er  than  0.02  lb/ in,  they  are  deemed  incompatible. 

The  above  can  also  be  stated  for  any  randomly  selected  pair 
of  springs,  devise  a  method  which  tests  the  diff  in  free 
length  and  stiffness  between  them.  If  the  diff  in  either 
is  greater  than  1/2  the  tolerable  deviation  from  the  mean 
(5.0  on,  1.0  lb/ in) ,  the  springs  are  deemed  incompatible. 
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Two  coil  springs  are  required  for  a  device  which  I  manufac¬ 
ture.  It  is  desired  that  each  pair  of  springs  be  compatib¬ 
le.  The  springs  need  to  be  5  inches  long,  the  coil  needs 
to  be  3  inches  in  diameter  and  the  spring  constant  should 
be  1.0  lb/ in.  The  manufacturing  tolerances  of  the  springs 
are  that  the  length  is  given  as  %  inches  +  0.1  inch,  the 
diameter  is  given  as  3  inches  +  0.1  inch  and  the  spring 
constant  is  given  as  1  lb/ in  +  0.2  lb/in.  For  a  pair  of 
springs  to  be  compatible,  their  lengths  should  be  within 
0.1  inch  and  the  spring  constants  should  be  within  0.02 
lb/ in.  The  diameter  of  the  coil  is  not  critical.  The 
given  manufacturing  tolerances  are  such  that  most  pairs  of 
springs  will  be  compatible.  The  objective  of  the  test  is 
to  find  those  that  are  not  compatible. 

The  spring  is  made  of  music  wire  and  its  operating  condi¬ 
tions  are  temperature  =  70  to  100  °F,  standard  atmospheric 
pressure,  current  relative  humidity.  The  spring  does  not 
operate  in  a  corrosive  environment.  The  test  conditions 
should  be  the  same  as  the  operating  conditions.  The 
springs  are  loaded  slowly,  without  cyclic  or  impact  loads, 
and  the  typical  deflection  ranges  from  0-3  inches  for 
loads  of  0  -  3  pounds.  Again,  the  springs  should  be  tested 
using  compressive  deflections  0-3  inches  (preferably  3 
inches).  Also  the  springs  are  restricted  to  weigh  4  oz. 
Again,  the  testing  conditions  should  simulate  the  operating 
conditions . 

The  restrictions  on  the  test  and  test  device  are  as  fol¬ 
lows: 

(a)  test  must  take  less  than  15  secs. 

(b)  the  tester  device  must  cost  $50  or  less. 

(c)  the  tester  should  be  ready  in  4  months. 

(d)  tester  is  restricted  to  weigh  8  lbs. 
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So,  concluding,  I  need  a  tester  device  which  weighs  8  lbs 
or  less,  cost  $50  or  less  and  can  provide  test  results  in 
15  secs  or  less  to  determine  compatibility  of  a  pair  of 
springs  and  I  need  this  device  in  4  months.  Also,  those 
springs  which  are  not  compatible  should  be  returned  to  the 
inventory  since  they  may  be  compatible  with  other  springs 
in  the  inventory.  Historically  only  6%  of  springs  are 
incompatible. 
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Develop  a  testing  procedure  to  determine  if  any  two  coil 
springs  selected  at  random  from  inventory  have  equal  per¬ 
formance. 

The  springs  are  4.9  -  5.1  inch  long  with  3  inch  diameter. 
Equal  is  defined  as  the  difference  of  free  length  of  less 
than  or  equal  0.1  inch  and  spring  constants'  difference 
less  than  or  equal  to  0.02  lb/in. 

The  procedure  should  allow  for  testing  500  -  700  pairs  of 
springs  per  day.  No  record  is  required. 

The  procedure  should  be  an  optimized  balance  of  lowest 
cost  considering  both  machine  and  labor  cost,  and  must 
satisfy  all  government  safety  regulations. 
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I  need  a  spring  tester  built  and  calibrated.  If  success¬ 
ful,  5  will  be  built.  The  test  bench  will  look  as  follows: 


S’  /  V'  5^V,yi*jS 


Design  criteria: 

1.  Bench  to  have  capability  to  evenly  load  both  springs 
(ie,  top  of  spring  press  must  be  made  to  travel  only  verti¬ 
cally  without  any  rotation) 

2.  Bottom  of  spring  press  to  have  some  means  to  locate 
spring  in  same  location  for  every  test  (recesses?) 

3.  Pivot  supporting  bottom  shelf  to  be  designed  thin  enough 
so  slight  rotations  of  bottom  shelf  can  be  detected  thru 
strain  gauges  on  sides. 

4.  Make/Break  on  rotation:  one  spring  must  not  be  stiff er 
than  other  by  .02  lb/in,  or  have  difference  in  free  height 
of  .01  inch. 

5.  Design  method  of  easily  comparing  strains  on  either 
gauge  for  quick  readout  (perhaps  light  on/off?) 

6.  Total  cost  of  bench  $50 
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A  testing  device  to  evaluate  the  performance  of  pairs  of 
coil  compression  springs  is  required.  The  springs  are  to 
be  used  in  a  device  that  requires  static  loads  from  the 
springs.  The  springs  meet  all  material  and  dimensional 
tolerances  for  the  intended  use.  The  detecting  device  will 
determine  whether  a  selected  pair  is  equal  in  performance. 
The  performance  compatibility  will  be  the  compressive 
stiffness  of  the  coils.  The  stiffness  must  be  1.0  lb/inch 
and  both  springs  must  not  vary  by  more  than  +0.02  lb/ inch. 
The  testing  device  must  be  designed  to  provide  a  compres¬ 
sive  load  of  up  to  10.0  lbs  force  with  a  stroke  of  not  more 
than  5.0  inches.  The  machine  cost  cannot  exceed  $50.00. 

The  testing  operation  will  be  performed  in  an  assembly  line 
environment  and  must  not  exceed  15  seconds  to  evaluate  a 
pair  of  springs. 
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E.2  Experimental  Data 
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LEGEND: 

CTL:  Control  number 

WORK:  Graduate  Student  (0)  or  Practicing  Engineer  (1) 

TPR:  Control  Group  (0)  or  Technical  Performance 
Requirements  Treatment  Group  (1) 

GDR:  Grader  S  (0)  or  Grader  J  (1) 
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E.3  Normal  Probability  Plots 
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Normal  Probability  Plot  for  Attribute  XI 
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Figure  10.  Normal  Probability  Plot  for  Attribute  X2 
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Figure  11.  Normal  Probability  Plot  for  Attribute  X3 
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Figure  12.  Normal  Probability  Plot  for  Attribute  X4 
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re  13.  Normal  Probability  Plot  for  Attribute  X5 
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Figure  14.  Normal  Probability  Plot  for  Attribute  X6 
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E.4  Three-Factor  ANOVA  With  Interactions 


Table  19.  ANOVA  With  Two  and  Three-Factor  Interactions 


Source 

Degrees  of 
Freedom 

F  Value 

Probability  of 
a  Value  >  F 

PFM 

1 

15.06 

<0.01* 

GDR 

1 

4.95 

0.04* 

WORK 

1 

0.56 

0.46 

PFM*GDR 

1 

1.02 

0.32 

PFM*WORK 

1 

0.67 

0.42 

GDR*WORK 

1 

0.07 

0.79 

PFM*GDR*WORK 

1 

0.38 

0.54 

♦Significant  at  a  =  0.10 


Table  20.  ANOVA  With  Two-Factor  Interactions 


Source 

Degrees  of 
Freedom 

F  Value 

Probability  of 
a  Value  >  F 

PFM 

1 

15.47 

<0.01* 

GDR 

1 

5.08 

0.03* 

WORK 

1 

0.57 

0.46 

PFM*GDR 

1 

1.04 

0.32 

PFM*WORK 

1 

0.69 

0.41 

GDR* WORK 

1 

0.07 

0.79 

♦Significant  at  a  =  0.10 
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